DOCOIEIT I^SQHE 



ED 159 211 

AOTHOR 
TITLE 
PUB DATE 
HOTE 

EDRS PBICE 
DESCRIPTORS 



TH 007 517 

A^dre", Thpmas ^ . . 

On Productive Knowledge and Levels ot/Questions. 
. [77] / . • .( 

- 77p. \ : ■ ' y • ■ . 

MF-$0.83 HC-$«.67 Plus Postage. 

J^dvance Organizers; Cognitive Ptocess.es; *Complexity 
Level; Concept Formation ; Concept T^achingj *Learning 
Processes; *Learning Theories^ Literature RevieiTs;^ 
♦ Memory; .*Ouestion^ng TechnoTigues; Retention; 
♦Taxonomy; Teaching Methods - 
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A m^del is proposed ^or memory that stresses a 
distinction between episodic memory for encoded personal experienlpe 
and semantic me^nory for abstractors and generalizations. Bas4cailf^, ^ 
the (iiodel holds that questions influence the nature of memory 
representations formed during instruction, and that memory , ^ 
representation controls the way in which stored knowledge can be 
used. The paper is divided into three major sections, 'in the f.irst 
section various proposed typologies of questions are described ^ to 
ident-ifcy the dimensions along whcih questions may vary and to ' , 
describe both the^ utility and inadequacieA of current typologies. It 
is argued that an* instructionally us.eful taxonomy must be based -upon^ 
considerat^-on of the empirical effects of questions, and that 
dfBvelopment of such a taxonomy is a fundamental goal of research on 
levels of questions. In the second section the limited research on 
levels of questions is reviewed', to in.t*egrate .what is known and not 
known about the effects of asking! different types of questions during 
instruction. Bo^th deficiencies in the existing studies and promising 
directions for future research are indicated. The third section 
briefly presents the information-processing model of knowledge 
acquisition and attempts to relate question-level effects to the 
^model. (Autbor/CTM) * 
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On Productive Knowledge and Levels of Oiiestions 
■ I, : Abstract • * 

'The .papof Is concerned with the effects of aj;kjng ^students qb^stions at 
different revels of cognitive conwlexity during learning. The first section 
reviewed taxonom i e^ of question levels^' and pointed out their deficiencies as 
research too 1 s ; • the-^cond section- rovi6wed research on leVels of questions, 
ft was showrt thaVhigher level questions can have facilitative effects on 
both reproductive: .W. product ivp kno'wlc^dge, but that the conditions under 
which suchTf.'icilLtat|bn occurs are not well understood. The third section, 
outlines an i nformat ion 'processi ng model' of ' Human cogni tio,n that can account 
for question level effects and that serV-o^s to integrate previous research on 
(|uestion level and to to provide, direction -for future research in this aVt^a 

V ■ ' ■ ' ■ ■ ■ ' . ■"■ ' ' . ' . ■ ' 



. On Productive Knowledge and Levels o^ Question^ 

/ Thomas Andro ' \. 

• ' / Iowa State University' , 

Educators clearly believe the type*^, of questions- teachers ask are 
important. 77ie first series of lectures on teaching methods offered at " 
Cambridge devoted considerable attention to the. art of examining (Fitch, 
1879). Most teaching method books since have discussed- questioning tech- 
niques /e.g^f^, -Burton, 1962; HoUey, ^1923; Hough, et al . , 1-970; Lancelot , 1929; 
Miller^ 1922; Otto, et al.', 1960; Ruediger, 1932; Shipley, 1972; thomas, 1927). 
Teacher training programs dutifully introdu.ce prospective teacher s to the 
varieMes and vag:aries of quest j^on-asking and exhort . teachers to ask 
questions that- rqquire thinking. F^ecent critics of American Education ' 
have condemned the question procedur.es of the typical teacher (Holt , 1964) 

Despite a century of pi^oletyzing for h^g-her- level questions, the. effect's 
-of questions on student Learning are ftot'well known. Empirical ' research 
examining .the effect;? of different types 'of questions on achievement has b^en 
rare. T|e conditioas under wi^ich; questions influence learning and retention 
remain unexamined and. unspecified. Although various typolQgies have been 
proposed, no general* taxonomy ' of questions that meets criteria of inclusive^"^ - 
ness and operatioaality has been widely' accepted . . Gi^en thii's state of affairs, 
educatio-n^l ^cl.a|ms for the goodness of higher-level questiop^^ remain wish-. 
fuLfilling myths. ' • - ., • 

■ ' . ■. / ^ ^ . ■ ■ \ ' ^ . ' , . . ■ 

.It might be argued that currpnt psychological ideas about the' Jlevel .or 
, depth of pro'cessing of informat^ion (Anderson, .1970; Craik^^ 5 Lockhart, 'l972j^, ' 
provide empirical support for a level-of-qiXestion effect . .Clearly research, 
on depth of pro^cessing. >as sli^wn that'/ within^ the types of processing that ' 

have^been varied,' a deeper level. of processing pro4uces better retention. . 

. ^ . >■ ..." ' ' v>-4^ ■ ■ • . 



i|owever, , such research provides at best 'circumstantial evidence for a level/' 



6 f^r^questipn effect. In all -bf. the level of processing literature, ^-compaTrisons 

have bqen'made between very primitive levels, oi processing arid procpssihg 

information for' meaning. Learning in Subjects who are, asked to irate son- . 

•tences for pronounciabili'ty pr imagery might be compared, for example. . Iir 

general, tihis research. has found that'-the more the inforinatioT^ is processed ' 

for meanirig^ the better it, is retained.. But the verbal leai:nirig-nieraory research 

has not compared different levels of semantic T^^ocessing and it is this 

latter comparison that is_central to the levels o^* questions .effe<:^t as vir- 

tually all questions that would be educationally relevant would involve some 

processing of meaning. Xhe educator s^! b elief that different levels o/ quesr 

tions differentially influence learning is^b^sed on thej^elief th^t processing 

mdre'-or-less deeply for meaning will produce differential effects on learning* " 

' ■ ^- - ■ *' /• 

and retention. This latter issue has not been examined in the verbal learning--^ 

memory research. « • : - - 

/ .' ; ■ ■ ' ' . ■ -■ 

The overriding purpose of this paper is to deal with' the question:'. What 

are' the effects on learning of different levels of que^.ions asJced durirlg 

r ■ ■ ■ ' ■ ' 

» • * ■ . ■ ' 

instruction? From this perspective the question is seen as a potential instruc 
tional |:ool that can be manipulated by a teacher or instructional designer in 
order to procjuce certain learning outcome^. 'The paper ,isvconceptuaIized as a' 
status repoi^^t on what is known about questions as an instruc'tiona.1 tool: It; 
is hoped^that the paper will provide g^n overview of current knowledge and a ' 
set of ^uidelirres for future research on the instructional use ^of quQstions^, 
The paper is divided into" three major sect ions . 'In the first section ^ 
various proposed . typologies of qu^jstions are^ described. The-purpose is to 
identify the dimensions along which questions may vary and to describe both ; 
the utility and inadequacies of current typologies. ' It, is argued that aa 
instructional ly useful taxonomy must be based upon consideration .of the empiric 



effect^ of ►questions and that development of such a taxonomy is a fandamental 

goal of research on'-leyels of questions., ^ . . . 

'> ■ ^ ' ■. , ■ ■ . . . ' ^ ■ ■ ' ^ a 

^" /^he se9ond section the limited research on levels ot questions ist 
■■*•*■, ' \» ' • >' ^ 4 

reviewed. The Intentioh''" is to - integrate what is known and not known about 

the effects. of asking different types of questions^during instruction. Both 
deficiencies in the? existing -studies and pi-omising directions for future 
reS^^arch are indicatbd. . * ' * ^ . - 

, * Finally^ .-the third section briefly presents^ a!i> information-processing 
model' Qf knowledge acquisition, and attempts to relate question- level- effects 
to the model. Basically the model holds that questional, influeivce the nature ^ 
of the memory representations formed during instruction and that the'memory ' 
representation controls, the way in which stored knowledge can be used. 
^Somc nefinit iohs / \ ' - » ^ . - ♦ 

As used in. the* present context,- c^ que^tion is a 'direction to a t^arn^r • 
to examine instructional mat^i^ial or his memory of it and to produce some 

.response. ^ Unde^r this conception,* bqth the statements, ''Memorize this poem* 
and recite it to me from'memdr^.''' and, "What isnhe capital of Iowa?" would 

.be considered qu'estior^s. .Jhis conception of, questions is necessary because 

* 4 * ^^^^ ^ ^ ' . ■ . 

•directive statements {imperatives) and. formal interrogatory sentences refer 
to eqil\valent cL/nitive And behavioral activities. There is little difference 
for example, in a^png "Who killed Cock Robin?" ahd s^ing 'JMemorize the name 
.of Cock Robin's assassin and tell me it when I say Cock Robin." The latter 
is not a formal question but would be included witMn the subject matter of ^ " 
this review. T|ie questions considered, herein are relatively^ limited in scope. 
Thus differences between inquiry, discovery, -and expository teaching 'arWifot 
eonsideted in this paper. Rather the paper is cbnc^rned with the instiMctiona 
effects of asking students to- make relatively short responses in the process . 
of acquiring«inew information,, skills, and knowle.^ge. . 
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• v. ,T|>e-'tcrms soKnition, cognitive process, fhcntal process, and thought J 
process are ^used so^yhat^ loosely and synonymously. They refer to the set 
of mental sl^-eps (i.W, program) that must be perform'ed in order to make 
response. Jhus they represent loose terms for the mental' pperations a 
hj^an 9arries out. in answering questions, br engaging in cognition., 
^ • Level ,of^ question refers t'o the nature of 'cognitive processing required 
to answer a qufestion-. A 'question may ask a learner to repeat or recognize 
somt> information exactly as i't was^presented in instruction. Such a question 
is\typically referred to as a knowledge, . factual , or verbatim question, 
f actual questions a^re believed to involve less complex cognitive processing 
than questions requiring more than direct memory (Briggs % Reed, ,1943; 
En-glish^ Welborn 5 Kdllian, .1934; Ausubel , 1963; Anderson, 1972) .\ ^ Quest ions 
that require more than^s^imple direct memory are believed to involiAe more 
complex cognitive processing. Describing the nature of such more-than-memory 
(higher-level) questions has been difficult, although a varfety of question 
classification schemes have been 'proposed. 

• Taxonomies of Questions ' . 

: ■ " • " . '■ — ^ " ' * 

Subst<-incd Questions . ' ' . 

, , In one early class-ification' scheme, Briggs and Reed (1943) and English, 
et (193'4), distinguished between substance and factual questions. Factual 
questions could be ans^s^ered on the basis of one text sentence; substance ques- 
tions .requii?ed-a. learner to combine information from more 'than one sentence of 
tjhe insti^tion. 'While Yhis scheme had the advantage of objectivity, in prac- 
tice i^ failed'tb discriminate, high and loW level questions. Given the text^ ^ 
rThe ball is red- The ball is round . 'V Jh6; question, **What/is red and round?»» 
vould bo classified as substance even though it .intuitively seems fairly factual 
iThe problem with Briggs ^and Reed»s knd English/ et ■ al » s scheme is that' separate 
Aominar sentences may be combined into oiiie' underlying proposition when they 
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are read.- Kintsch (1974) and *Gr.eeno and Noreen (1974) have presented 
yevidence that mature readers normally combine sentences this way. ^ 
Meaningful Questions ' * . 

Ausubel (1963, 1968) distinguished betweeh rote and meaningful learning. 
Rote learning involved' memorization of material in a verbatim form. No attempt 
was made to relate the new information 'to previous ei^periences . - Meaningful 
learning involved rlon-arbitrarily relating tO-be-leat*ned information to 
previous knowledge. Knowledge was contained in a hierarchically organized 
cognitive ^itructure; neV/^ knowledge, wds meaningfully learned when its place 
in the cognitive structure was determined and^it was subsumed into the struc-^ 
ture. UnfOrtunaAiely , Ausubel failed to provide objective guidelines for the 
generation of meaningful or rote questions. In fact, in much of his research 
on supposedly meaningful learning,, the question^ used sQem tOv have been pri- . 
marily factual (Ausubel 5 Blake, 1958; Ausube-l^^ebbins^-^ -BlaW " ' 



Ausvibel, jBtager Gaite, 1968; 1969). 
Taxonomy of 'Educational Objectives: Cognitive Domain 

A variety of more complex classification schemes have been pro^t3^d. The 
best known is probably the Taxonomy of Educational Objectives: Cognitive 
) Domain, developed by Bloom and his associates (Bloom, Engelhart, Furst, Hill, 

and Krathwohl, 1956). This Taxonomy /described six general classes of behavior 
I that might result from instruction.' The six classes were: 

Knowle/Ige : This class esentially involved repetition of information in 
the form it^was .presented. This kind of behavior is called for in. what I have 
labeled factual questions. 

Compreliension : This class essentially involved recognition or -production 
of some paraphrase of material presented in instruction. . 

Application: This class involved use of presented information in some 
new s'ituation. It could include recognizing new examples of a concept or 
ujving a principle'in a problem solving situation. 



Analysis^ Behaviors in this class involve taking a* given situation and ' 
decomposing that situation intp^^its component parts and their relationships. 
Ty|j^)ically this ^involves using sbme previously taught, scheme to decompose the- 
whole.. For example, the student may'be told to analyze a short stoyy. He * 
would produce descriptions of tKe main characters*, mino^ charact^ers,, plotj" 
theme, £lnd 1 it.erary style. . " , / . 

Synthesis; Behaviors in this .class involve production o£ some product 
given appropriate elements. . Writing a'short story,-'mighi;be an example. 

Evaluation; this^ category of behavior involves judgments about the value 
of information, concepts, ideas, relative to s6mc goal or purpose. ' ^ 

Seated iB such general terms, the classes may not seem particularly us.e- 
f^l^:-^|j?0Wever, Bloom et al^ identified subclasses within each general class . 
and gave examples of each type of behavior. The.compreh|)hsion class, for. ^ 
examptcT- was- divided-rnro : - trarns^r«^^^ 

of presented material, and extrapolation from presented material. However, 
there was no empirical verification of tH|^^c^t^gp^ies and no attempt to demon- 
strate that 'the cognitive processes presumed to operate at each level were 
psychologically as well as logically distinct. " ; 

•'The taxonomic levels were assumed to be hierarchical; cognitions at the 
higher levels, presupposed the ability to carry^ out lower .level cognitions; one 

could not comprehend specific content unless one kflew it,' etc.— The levels of 

I ; ^ ■ . . 

the taxonomy varied only along a dimension of complexity. However, ^adaus, 

S ■ ' ' ' - 

Woods, and Nuttall (1973) examined the hierarchical nature of the Taxonomy and 

concluded that the hierarchical structure was questionable and that^>erformance 

on each of the levels strongly reflected- general ability, ... 

^ t^espite their detailed elaboration and wide use, the Bloom et al Taxonomy, 

afi^l^others based on it-, are not satisfactory classification schemes.- One 

V yH ■ ■ ■ ' ■ . . 

' ^^^^^j^^problem is that the' distinction between the categories tends to become. 
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blurred in practicev Cons ider^the following: a studej^it is given a verlfel ' 
des-cription in. which a cdt is. given a f;ood pellet every fourth time the caf 
presses a lever. He. is asked a mAltiple-choice questioh about the schedule 
of reinforcement. The correct answer is fixed ratio scjiedule. Thi;s question 

.might be clkssifie'd at the '•application level as it is' asking tjie student to 

'' ■ ■ . * « .' ' 

recognize- an , example of & prelsented cojicept. However ,- .if the example was 

presented to the -stmient during" instruction, t^ieii the .question- is factual or V 
at the .l^nowledge' level in the Taxonomy. Bioom'et al (i956) recognized this 
relationship b^w'een the level of a question and the instruction material, 
but many us^s of the Taxonomy have-not. More seriously, assuming an unfami- 
liar example, the q^aesti9ns above 'could: ^Isd conceivably be classified at the 
•■ analysis level. Is not the studept asked to identify the elements of the 
reinforcement situation arid their relationships, to determine the nature of 

_reinf orcement ? . Many -quest^i_ons-so defy- antique -c^lassi-fieaticAi- within the ' — 

Taxonomy. ' ' 

The basic problem with Ta6conomy-like schemes is' that the procedures for 
-assigning questions to a-level are not sufficient ly detailed tp be entirely' 
objective. Assignment" of a question to a level very af^ten requires , a professional 
judgment, there is little evidence on the reliability of such judgments. 

Evidence from classroom. observation studies su^ests that inter jmdge relia- 
bilities tend 'to:fee ^moderate (Dunkin ^ Biddle. 1974), However, such studies 
have examine/ only the reliability of classification of questions that teachers 
^ actually ask, not the classification reliability of all possible quesit^ions. 
Since teachers' que.stibns ai^e predominately factual the reliabilities may -not ' " 
bcf representative .for classificatio^ns of higher level questions. One se- " ' 
quentc of this lack of. detai l.ed rules for assigning questions is the di iculty ' 
in writing questions to match particular level . No dperatipivil, that 
unambL^^^uo.us and mechanical, rules exist that woul^ allow researchers (ana 
'tcachcrsJ ) to 'write questions for a given level., 



Structure oi; Intellect / * 

• The problem of classifying questions is equivalent to the gerieral problem 
. of ^lassifying cognitive behaviour. Any taxonomy. df fcogrfition can be. used to 
classify questions. One 4n|>^>fitial taxonomy" of cognitive behaviors iV'desl 
cribed by Gyilford's Structure of Intellec't. model (1966. 1967). Guilford's , 
model Hi^f^s from^J^loom Taxonomy -in. tt^major ^respects . - First the Sdl 
mod^l postu^ep^at cognitions differ in three dimensions Ccontents. pro- 
^ ducts, and operations) i,vstead of the single dimension ^.comple^dty. . ' ' 

• Secondly., oven though logically derived, the model enjoys some empirical suppor 
from the factor analytic studies of Guilford and others (Guilford^. 1^67).''' * 
. Ac:cording /o the model, an intellectual activity- or cognition ca, be carried 
jrat on on^::.of four contents: figural (ima^i'nal. spatial)' symP^lic. sgptic, 
f or behavioral. The activity can involve one of five types of operations! .(or 
prQcesses),;^cagnitiGn^memory^divergent-productimT^n^ 



and evaluation, and can result in one afsix types of end T,roiicts : units, 

classes, relations, -systfefns. transformations and implications. .The resultant 
' three, dirnferts^onal array yields 120 hypothesized mental abilities.". Since the 

Guilford model is well known, it^ 'further elaboration i^ not necessary here.' 

Prdm the present perspective the-value of the SOI model: is that one could pre- 
.sumabiy write . , questions that would engage a given mental ability. Thus, the 
model could serve as a question taxonomy. ' 

-.Aschner and Gallagher (Aschner, et al. 1965^ Gallagher. 1965) attempted - 
to develop a classification system based .on Gukord' s model and to use it to 
study teacher's in-class questions. Gallagher \^965) reported considerable 
difficulty in .classifying questibns reliably within thfe categories and 
Dunkin and Biddle ^ (1974) characterized t,he reliability of question classifi-,; 
cation instrumeiits based on Guilford^ scheme as y^eakpr than' those based on ' 
Bloom's. On6 practical difficulty may 'be that questions typically asked by 
teachers ca 1 r;intb pla> several, mental operations and^ abi-f ities . If so,' 



unU\\je class i fic^ty:)n of ' quGs'tipns into one of the '120 cltefepri'es vyould be 
impossible. Thus, like the Bloom Ta^^onomyv/ no schemd bis^d on Guilforcl' s- 
mo(Jlcl un'amb iKUOusiy map*i questions to categories. ' 

Conditions of Learning Typology : ' ^ ' • , " * * k^^^- 

^Gagne's hierarchy of types of learning (Gagne, 1970) > has relevance for 

the problem of. classifying questions.- GAgne's schehie distinguished between 

eight types of learning: signal learning, stlmulus-i'esponse learning) * chaining * 

verbal , association, discrimination leilrninig, concept learning, principle 

learning, and probie^n-^iolving. These types oT learning differ in the. conditions 

.necessary for them to occur. Such conditions were identified .by exajpining the 

experimGnt;al paradigms in which the various types of Icarning^ had been studied. . 

i \ ■ <^ 

Since Gagnc*-.s scllenfe' is well known it will not" be described in greater detail ' 

here . • . ^ . ' ' . • 
jjyi)othet ica Ily, it wpuld be possible to write questions to assess differ- . 



eijt types oi^.leaming. Clearly, it would be possible to wrTte~~arqTOslHrari--to- 
assess a learner's ability' to . make a discrimination, or to recognize a n6w 
in;5tary:e ►of a concept . •• Thus, a, questibh taxonomy -scheme based on Gagne^s 
model^would have certain advantages . The*'fact - that levels in Gagnei's model 
are defined in terms of the. environmental conditions df learning might' be 
helpful in developing objective crj.teria in- classifying questions. In addition, 
it would be possible to relate instructional questions to l^he ex:p^ected ouf-^, 
comes of , the instructional system, since the Gagn^. model presumably: encompasses 

■ ' ' ^ ' ' ' ^ V ■ ■ 

,the universe of coghitive ins.tructioLnal^ outcomes Some disadvantages of the 
Gagne model are that sonie types of .questions that do not readily fit withinthe ' 

model^ o^\ ., questions Bloom would call evikluative .-for" Vxample..' Also it does 

« • ' * ■ , ' • ' * ' It"" ■ " • 

not seem possibW to write questions for the simplest levels in 'Gagne^s hierarchy. 

Although the author is unaware of any classification scheme that makes 
use of Gaj:j^ic* s full, model, a classification sche/ne proposed by Anderson' (1972) 
^ittempt^ to Of)er»ational i ze at^east two of Gagne's categories/ Anderson (J972) 



di^^cussQcl the prbbj^s^ mcasurin^^ student comprehension and presented a 
pos^si,ble classitication scheme o^ items us,ed to. assess Comprehension/. 
-•AndQxson argued that ^lie first factor to be considered in determining /the 

;p,lasslficati9Ji of quQ5tioas,was thd relationship, between the' question and the 

• \ " .9"^ " " ■ ■ ' : ^ , ^ ^ \ 

prestente\l instructional matei^ial, Questions could be" presented iiysither 
the same lexical form as in the original instruction (verbatim questions) . 



yi a chan^^.cd IcxicalN-foxim (paraphrased questions), And'er.son ^rgued that 

verbatjm 44uestLon.s could ^ften bo answered -^without the student understanding 

^\ ' . • » " ' 

the material / Verbat im qut?stions correspond tj^ what I have cal le^ factual 

<luestions ajid. wfold correspond tis^^Gagne » s verbal asso^ation • learning . Parav 

phrasei^^qucstipns involved chapgiifg the cont^^ w^rds of the instruction' in / 

pr^estJntinj^ the question.^ Bormu^t^ (1970) made^'ii simi lar 'distinction. Anderson 

clearly recognized that the verbatim-rparaphrased distinction formed a continuum 

. . - ■. ■ . . ^ . ■ ■ ■ 

^^ven question nas near and distant paraphrases; Anderson argued, however, 
•that a' correct answer to any paraphrased qu^stioii ensures that ''the' student has 
proce^s^d the meaning o the presented information. , In Anderson^s words, 
answVriog a paraphrased"^ questpn presupposes "semantic encoding;*' 

"Aiulerson further argued^ tha1;^^£Qj;>^mcepts and principles 'it was possible V 

to write- application questionsT Such application quest loas"i'eqtiired the stu- - 
/ . " ' , _ 

-uent to/rccogiuze unfamilia-Y new Examples oi C-pucepts or to use the principle 
to sol,ve an/unf.'Jmi liar problenu Only if a student could answer such qu^i^tions 
^ould he he snfid to understand the concep^t or ""principle (Anderson, 1S)72; 
Anderson l\ 1-aust , 1973, Chap. 10). Oagne (1970) presents similar arguments. 

Within its nmited vniyerse of apP 1 » cab L l,i,ty , Anderson's c lassl f ication( '/ 
scheme seems to provide ;hi objective means of categorizing ciuestu)us. It is 
^tntaiuly possible to ''ol>j ec t i ve I y determine whether content wol\l^v are repeated 
both ill instruction and in\iuest idns . Similarly, it is possible to^tell if 
^ui vxample or |)r<)l)lem used in'an application cpiestLon was also used in the 
instructional iiuiterials. listimates of the closeness of instructional and 



test-item examples can be empirically determined^ (e. g. , Anderson, 1973)* 
This makes Anderson's scheme very useful for many/research pui;poses. The 
major fauit of Anderson's scheme is that it does not encompass many of the r 
se^^ingly wide varieties of questions. Questions at Bloom's et al analytical 
synthetic, or evaluative levels are^ not included, for example. Certainly 
these levels appear to represent important behaviors that might result from 
'instruction. ' In feirnes3, it should be noted that Anderson's scheme, was ^ 
developed in thg^ontext of a paper describing means ' for' assessing compreh^sion 
of concepts and principle's, not' as a universal scheme for representing instr^ac- 
tional goals or questijDns. Its laipk of breadth doet not obscure its valuable 
contribution. 

Towards an Adequate Taxonomy * . , 

This brief overview of question classification schemes should make it^ , 
suf f i"^ iently clear that students can be asked to do a bewildering variety 
of things with instructional materials. This wide variety makes it difficult 
to agree on any particular organization scheme or taxonomy. Thus, while a 
wide variety of question taxonomies have been prof)Osed^ all seem to suffer -i 
deficiencies of either objectivity or breadth. As will be seen in the next 
section, this diversity of taxonomies has led to little continuity between . 
various empirical studies of question level. 

In general, ([uestion taxonomies have been developed on a logical a priori 
basis. The taxonomist has postulated one or more dimensions along which ques- 
t ions, vary and then attempted to develop descriptive categories within those 
dimensions. Different taxonomists have emphasized different dimensions and 
thereby produced different taxonomies, fjuch taxonomies have been characterized 
by the use of subjective judgment in the classification of questions and have 
typically in prac'tice been at best moderately reliably means for classifying 
({uestions (Dunkin r, Biddle, 1974). 



Until some degree of uniformity and agreement is reached on a taxonomy 
of question^) .research on the instructional effects of questions Will remain 
fragmented. The development of an agreed upon taxonomy is extremely impor- 
tarit 'and has impact beyond the' mere description of questions, adequa-te 



taxonomy of questions would also serve as a taxonomy of learning outconles 

)n. 



and would provide a mechanism for assessing various outcomes of instfuctipn. ^ 



, One alternative approach to developing a useful taxonomy of questions 
might be to attempt to identify as many possible dimensions along whicll^ 
questions piay vary and then to use empirical ipethods to form categories qr 
clusters of questions. The current overview of question taxonomies suggests 
a number of siich" dimensions : complexity of processing;^ na,ture of product, 
operation, or content; type of learning involved; and, perhaps, subject matter 
A set of questions differing along such dimensions cbuld be d^v'^eloped and 
numerical taxonomy and clustering methods used to form 'groups of questions. 
Such groups could then be used to determine if*they in' fact produce varying 
instructional outcomes. That is, the instructional reality of the taxonomic 
divisions^ could bo assessed. Obviously this suggestion envisions a long-term 
program of pjirametric research on questions. Such a program would be both 
expensiveiv^c^arry out and frought with practical problems. However, until 
such resparch is available, an adequate question taxonomy and an adequate 
description of Instructional outcomos is not possil)le. 

What would an adequate taxonomy of questions consist of and provide? 
Such a taxonomy would have three major features. F-irst, it would provide for 
the relatively objective classification of c|uestions and would provide an ^ 
objective mechanism by which questions o 1: various types could he constructed. 
l\yr **oh j ect i ve** is meant that the procedures for classifying questions would 
he as iiKJchanical as possible and would involve a minimum of judgment. Ideally 
the procedure for classifying cjupstions (or constructing (questions) could 'be 
computerized. In the absence of such automation, the interjudge reliabilities 



for the classificaetion of ^estions should be very high. Inherent in this 
characteristic is the necessity for the taxonomy as a whole to be inclusive, 

. while the, individual categories sjaould bg mutually Exclusive. ^ Any given 
instructional by i:elevant question should be uniquely categori^zi^ wit-Kin, the 

, system. Given the ambiguities and cgmplexities of English, it may be^ tjiat 
some ^estions would need to be decomposed into subquestions which could be 
uniquely classified or that 'context must be used as a clue to the nature of ^ 
the questions. /Ideally, the pVocedures for decomposing questions or for 
utilizing context would al^o be mechanical. ^' 

Secondly, ' in an adequate taxonomy the categories of questions W^)uld be 
shown to have psycholpjgical and instructional validity. Empirical evidence 
would be available that the psychological processes involved in answering the 
questions and/or' the i^nstructional outcomes of asking the question during 
learning would differ from other categories of questions. Unless some psycho- 
^logical or instructional difference between categories can be observed^ classi 
ficatidp of questions into different categories is meaningless. One conse- 
quence of classifying questions by their effects wouTd^ be that the taxonomy 
would be instruct ionally useful. Such a taxonomy would indicate' what sorts 
of instructional outcomes might occur if various types of questions were used 
durin)^ instruction. The taxonomy should indicate the probable instructional 
consequences o f aski ng varying types of questions during instruction. 

Finally, an adequate taxonomy would indicate how questions relate to 
other instructionally relevant variables. Such other ' variables might include 
Individual difference variables such as age, cognitive style, intelligence, 
etc.; and instructionally manipulable variables such as feedback, pacijig, 
re in forcemeat , media , etc . As will be indicated in the subsequent section, 
there Is evidence that the effect of questions .are modified by such variables. 



Currently available taxonoihies falj far short of this description of 

an adequate taxdnoiny'. Taxonomies such as Bloom^«, Guilford's and their/deria- 

. • -y y ^- ' • . 

tives are ^est >wBgarded as heuristic in value. They remain sources of ideas' 
for ways in which 'i^ues^tions may vary and a starting point for research on 
'question taxonomies and ^the effects of questions on learning. The subsequent' 
section describes currently available research on tKe pFf^tts of questions. 
The majority of such research has used the. taxonomies described above as the 

starting point of the investigations. . - ^ 

' ■ ■ ■ . ^ * 

Empjirical Studies of Question Level . ^ ■ 

Questions may be lis^^ in at least four different si tuat ions ' to guide , 
student learning. In one situation questions are used in classroom recita- 
tion or discussion. ^ A sei^ond use involves questions inserted in text or 
other instructional media. A third use, in some- way? similar to the second, 
involves the questions used on examainat ions . Finally a fourth situation 
involves questions students ask of themselves while studying. 

The literature on oral recitation questions has received a number of 
excellent reviews' i^ecently (Gall, 1070; Dunkin Biddle, 1974; Rosenschine, 
Note B) • In general these reviews have concluded that the degree to which 
teachers use higher-level questions has little demonstrated relationship to 
student achievement.* While this conclusion is discouraging to those who 
believe in efficacy of *higher-level questions, its impact is tempered by 
methodological and conceptual problems which. beset such observational 
research. One serious problem is that, as noted above, the reliabilities 
wrth which teacher *s fjuestions can be classified are at best moderate 
(Dunkin T, Biddle, 1974, Chap. H). A second problem is' that teacherVs oral 
questions arc typically answered by only one student at a time. Anderson' 
and Faust (197.'^, Ch'ap. 6) have argued that a requirement for a stuaenr to 
make a response will not have an instructional effect unless the student 

ERIC 
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actually make^^a response. To the extent this generalization ^is valid, a^ 
question in an oral recitation situation provides, benefit to only one pupil 
(perhaps with some ional benefit to*those students who respond covertly) 

(< related problem is that the classroom research has not typically sepai^ted^* 
the effects of question^ on the acc^uisition of material covefed in tffe ques- 
tions and material not covered by que'stions. A$ the research bfelo^i^lvill 
indicate, the effect of' higher-level question^ s^ems to be specific to tuie 
concepts, principles 0^ ideas covered by the questions.- 

These problems lead me to conclude^ ihat observational research on 

, teachers in class questions is not ai) appropriate vehicle for systematically 
examining the effects of higher- iQvel questions on student learning. ^' For 
this rei^son, and also because o'f the quality\9f the recent reviews , this' 

> line of research is not ^considered further her.e. . ' . 

Also. not discussed is research on tKb effect of student generated - 

questions or examination questions . Th^se areas are excluded because res(^eajrch 

-is not available, although Frase and Schwartz' (1975) have initiated som^ 

r * ■ 

interesting research on student generated questions. , 

The present paper focuses o'n experimental studies which use the adjunct 
que^stions technique popularized by Rothkopf (1966). The 'adjunct questions 
technique involves placing questions ^either in, before, or 'immediately after 
prose passages and asking students to answer ^uch ques^tions while studying 
the passage. Later students are given a posttest whicii may repeat the adjunct* 
cjuestions and may ^Iso include new questj.ons. When tl'e adjunct questions are 
Interspersed in the text, they are pften called insertied questions ;^ I will 
adopt that usage here. No special term has evolved for adjunct questions 
massed cither before or after a passage. Rothkopf was'iWt the first to use 
adjunct' questions, but most contemporary research on cjuestions derives from 
his' 1963 and 1966 papers. Farlier work by Germane (1920), Distad (1^27], 
. Washbourne (1929) and Holmes (1931) did not generate sustained interest Ixnd 



in fact /became generally known only after cytTvent interest in RothkopiLUs^ 
work hacTbloomed; ^ , ' . 

Rothkopf s seminal studies have engendered a large number of experiments 
examining adjunct questions. Anderson and Biddle (1975) provide an excellent 
review' of this general literature. ' A less (detailed review is ^provided in . 
Faw and'^Waller (1976). Relatively few of*the adjunct questions studies have 
examined question level; ./it ijs, those studies that are reviewed here. In. the 
typical study* examining question level, students- are asked^o read a passage 
and to answer one or more, questions about the ; passage while studyi^ng it. 
Subjects'in different groups are .given questions O-t different cognitive 
levels, the "levels being defined and chosen by the experimenter. After 
subj.ects compi<jte the passage they 'are given a posttest on the p)assage. In 
various* studies this posttest has contained items asking for factual recall, 
items asking for higher-level processing, or both. Compai;'i^iv6f.^erformance 
on the posttest has been the variable, of main intel'est in this ' reisearch. 

Research using the adjunct questions technique to examine qites^ion level 

effects can be divided into two classes on the .basis of the type 6f posttest. 

employed. About half the studies employed a pos;ttest that asked «nly fpr 

factual or verbatim recall of the passage. The remaining studies empi<i^d . 

a posttest that contained higher- level questions. It is argued below that 

-.the first class of study can only yield ' results that are of best minor interest 

to inatructional theory . '-Moreover , it ^secms likely that different psychologi- 

♦ -> , 

«. <•'* 

cal mechanisms arc needed to explain question level effects in the two types 
of studies. I-or these reasons the two classes of studies are discussed 
sepai citely . ' 

Studies Us\ng 1-actual Recall Tests ^ 

In an early study Rothkopf and Bisbiscos (1967) varied the ncHure of the 
content material required to answer questions. While not directly concerned 



;with the effects of question level, the study was interesting because it / » 
demonstrated that%iuesticn6 could influience how. the students processed the 
material^. Some suT:)jects rec^^^d questions that ret^uired proper names or 
numbers as answers; other subjects received questions tha^ required either 
common words or technical terrns as answers; a third'^and fourth group of 
subj^cfs received slither all types ''of questions or no questions respectively. 
All -groups took a posttest containing all question types aftet reading.. The 
subjects given proper name and .number questions did well on that class of 
material on the posttest , .but pob^y on other; material . * Subjects ^who received 

. ■ . ' '■/'.■' \ ' ^ : • * 

commorl and technical word adjunct questions, or mixed adjunct questions, 

did well on all types of posttest/questions . Rothkopf and Bisbiscos 'attributed 

their findings to the influence of the questions on the student's attention 

r 

to the materi^al . Subjecf^s who received name and number adjunct questions could 
limit their attention ro material of that type. Subjects who received 'mix[d ^ 
question types had to attent to all types of material in the passage, since 
"'common word questions is a broadly defined class. 

Frase (1968) . demonstrated a similar effect of questions on attention. 
Trasc varied the sp.9ti^Pi^ity of the questions. Consider the following taken; 
from Watt.>/ (1973) A. 



'John is a painter' and is 25 years old. Bill is a carpenter 
^and is 64 years old. Sam is a 'mechanic and is 30 years oid. Ed 
Is a Writer and is 70 years old. 

The ({uestion *'How old is John?*' is more specific than '*How old were the 
men in the story?" ^^omparing , specific to more general questions, Frasc found 
that the broader ' questions led to poorer posttest performance; a rc^sult 
seemingly at odds with the Rothkopf and Bisbiscos findings. Watts (1973) 

resolved the apparent paradox by demonstrating that Fras^^ls broader questions 

■ \ 

did not re({uire the student to attend to the associative relationships that 

\ ■ ■ 



I 



were tested on the posttest. On the poS^test, Frase ai^ked questions like 
"HqW old was John? HoW Qld was Sam?'* . Sj^bjects given the broaciery posttest 
questions did not have to relai;^ names to ages during learning and thus 
did pooi'ly on the posttest. When Watl^s (1973) .gave fifth graders broad 

questions, that -forced them to attend to posttest relevant relationships, 

') ' \ ' 

achievement was greater than , in ••a/*' group given- simple .factual questions on 

. / • ' ^ • ■ . ' - 

the material. Watts- (1973) used bro^dej: questions like "Which man is too 

■; ' : ' ■ V- ' ' • ' . ' ^ ' . . . 

old for his' job?*' Such a quest ion^learly seems highe^' level sifice it 

requires the student to make inferences and to compare the ages of the men 
to expectations about the amount of labor each job^^Juires and the physi- 
cal .condition of the men. (Bloom et al , 1956, would probably class^'ify the 
question^ as an evaluation level question.) Unfortunately the Watts (19>3) 
study does not allow us to proper ly , assess the effects of these higher level 
questions since the posttest merely tested for factual knowledge. About all 
•that can bj(^said is that the higher- level ques^tions led subjects to remember 
more facts , from the passage." 

A similar problem exists in an ingenious ^t"|yy by Ric^ai>(is and DiVesta 
(1974). Subjects read passages containing a topic sentence that asserted a 
gqneral (quality about a fictional country called Madaand three sentences 
wl^'rclv supported that general assertion. The relationship between the topic 
sentence and ^he supporting sentences was not made explicit'. Thd'^paragrapli 
below provides an example. ' . 

The southern area of Mala can best be described as a desert. 
Rainfall is less -than two inches per year. The soils in the 
southern area of Mala are either rocky or sandy. In the summertime 
temperatures have been recorded as high as 135 degrees in soutljern 
Mala (Rickards ^i ' DiVesta, 1974, p. 355). 
Ileaders of these paragraphs were either ^sked rote factual cjuestions which 
required repetition of one of the three supporting sentences, rote idea 



questions which required repetition of the- topic sentence, or meaningful 
learning questions which required the use of the supporting sentences to ^ 
justify the general assertion. An example of the lattei^-type of question 
•i^ *^ Why can it be said that 'soiithern Mala is a ;desert?^' A fourth group 
^received inserted questions that were irrelevant to the passage. When ^ 
questions were inserted after ev'ery two paragraphs, m\^aningful learnings 
questions led to superior performance; however, this effect disappeared when 
the qfuestions were asked ^fter ever^ ^ur paragraphs . Perfoi^ance in this 
case refers to verbatim recall of. the passage. ^Subjects given meaningful^ 
learning questions recalled more. Thi^re^;kl& is congjuent with Frase 
(1968) and Watts (1.973), since in order to answer .the meaningful learning 
question the. subject would have had to attend to more of the passage, 
having attended to more, he recalled more. In aperies of studies using 
similar m'eihodo logy, Richards and his associates have confirmed these basic 
results (Richards, 1976a, 1976b; Richarcls Hatcher, 1976, N(gte 7 ). 

Allen (1970) also repotted an apparent attentional pf feet . AlTen comVr^ 
pared groups, given either memol^y level or higher level questions during j 
reading. His posttest appai;ently consisted of factual 'questions dealing j ) 
either with material re'fered to in the memory level or higher level adjunct 
questions. Ihe basic result was that subjects did better on questions related 
to the information they were asked about in the adjunct questions. This 
result is consistent with the hypothesis that question^^ serve to focus tbo 
students* attention on particular material. Like the FrasQ/(1968) and 
Watts (1975) studi es , neither the Richards and Di Vesta (1974) . nor Al Icn (1970) 
studies demonstrate any effect for higher level questions beyond directing 
student's attention to more of the information. 

A series of studies by Frase and his co-workers made this directed attention 
.hypothesis explicit, Frase ' (1969a, h, 1970a, b, 1972) had people read 

passages that assorted a series of clas^ inclu;;ions. The structure of the 



passage^^ was: A is a member of B, B is' a member of C,- jC is a member of D, 
D is a/member of E. Heading each passage was an assertion that could either' 
be pr^Oven or disproven from the information in the passage. The subjects 
were told to read^the passage and either verify or disprove the assertion. 
Later tiley were asked to recall the passage and to draw any conclusions they' 
could from the passage. i 

Fras^ varied the level of the to-be-verified assertion. An assertion 
given in /text was the lowest level (Is A ^ .member of B?) , the next highest 
level involved relating two classes ^separated by one class (Is A a member 
af'C?) and so oh. Frase predicted that factual recal 1 of the passage would 
increase as the level of to-be-ver if ied assertion increased, but that the 
sentences rpc^led would be those^needed" to , verify the : given assertion. 
Oveythe series of studies this hypotKe^is was-confirnied. , 

Fra^c' had also hoped that . verif>^ing higher level assertions while study- 
ing. would lead readers to draw such ''f^^ferences during testing. This pr-edic- 
tion. was weakly confirmed. However, the overall level of inference drawing 
during recall was quite lo.w across the series'of studies, never exceeding,^ >- 
"15% of the possible inference,s\ Additionally, ^^e inferences drawn during 
recall tended to be those needed to verify the' adjunct assertion, 'Similarly, 
'F-rase and Si ibiger (19703 led subjects td read a greater or ' lesser number of 
text sentences tO' soiye an adj unct .problem.^ Subject*^ who read more of the 
sentences remembered more. However their increased; retention was specific 
to the sentences needed to solve the problem. 

^TwQ methodological problems may have influenced la-ase * s results. One 
problem involved the directipns given to. "readers on the recall /tests . I'rast;^ 
simply tQld subjects to draw as iftany inferences as they could from the., 
material. Assume a subject in Prase's experiments wrote: A% ^^r-;^;, Bs Bs are" 



Cs. The inference that As are Cs'is obvious, so obvious, th^t' Ir may have not 
been worth the subj ccts ' e^ef fort ^to wi^Lte it in the ab.sence of clear di j'ect ions 



to do so. 



ftf addition! the specific cl^irj^-inclusionary relationships used.by 
Frase wfere npt made clear ,in his passages. Subj ects could have interpreted 
the relationships as equivalence instead of class- indusionary relationships. 
For. example, one passage assefted that- tlf^Fundalas were outcasts. What 
was meant was tKe luindalas were one of several tribes of outcasts, but it 
would be ea^.;^^^^^^^ the passage as asserting that Fundalais is the 

nanie given to all outcasts.. Griggs , (1974) provides evidence that subjects 

did in fact make such interpretations of materials, 1 ike l-rri^eVs. To the extent 

■ - . J. ■ >^ ~ d 

that subjects did assuijie equivalence relationships Frase' s scoring procedure 

> *,'.■'• 
underestimated the degree of inference drawing. ' > '. 

While the e^^tent to wh\ch these methodological; problems may have influ- 
enced Frase's data cannot be ascertained, it should be noted that others 
researchers have found that subjects have difficulty correctly drawing the 
kinds of class inclusion implications Frase required (Griggs^, 1974). .Th\is, . 
the low level' of perf^^jmance in Frase study may simply reflect the inherent 
difficulty of the task. • . / . 

In any case, Brase's data strongly* support therct t:ention that one 
effect of higher level adj unct 'questions is to dircn students to attend to 
more of the material and, thus, to recall information diVectly related to thp 
information needed to answer . the questions : Such an effect of questions may 
he. labeled the dii'ec1;ed attention effect . Underlying the directed attention^ 
effect is the principle of least effort. As applied in the presen'^ case, the 
principle of ^eftsf effort kuggestt^ that; when students- are confrontred with - 
more ' Information than they can comfortably assimi late,, they wi 11 selectively 
ifttend to those aspects of the information needed to conijilete their iiercofved 
task of ^getting t|)rough' the material. Adjunct . aids such as iasertotl qvio^'- 
tions, serve to direct the" .sttidegits attention by altering the nature of his 
perceived task. Tlus exi^lanation of the effects of (|Uestions may be called 
the directed attention model (DAM). 
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DAM ifldkes it possible to relate> the <juest^,n level" literature discussed 

to t^is paint to the general literature on adj unct -instructional aicj|. A ^ ' » 

^ \ - ■' 

variety of such aids have been shown to have effects, that can be accounted/ 
for by DAM. F03? exampli^, Anderson § Faust, (1967) and" Anderson, Faust 5^ 



Roderick, (L968) demonstrated that "a formal prompt in programmed ■ instruction 
• < - ' ■ , ■' •■>✓, • • 

could lead Students not to^ attend to associative 'relationships^ that were 

— ; , ^ ■ ■., • , ,. , ^ I . ■ . ^ . 

tested on the posttest. Subjects who received th^ formal prompt performed 

' .'■ \». . , . ' . ' ' . „ " ■ 

more poorly than subjects who dijd not. This, result is^ similar to the Frase 
(1968) result involvii^g breath of questions. Like Frase' s.ciuestions , Anderson 
and Faust's prompts served to alter the subject's peilcepti6Ti of his task 
4\nA limit the material to which he attended. Similarly Anderson, Kulhavy, ' ' 
and Andre (1971) found that.^when ,subj ects could examine -feedback' in>pVogrammed 
instruction prior to reading, the. frames they tended, to simply copy ' answi^rs 
into the blanks and not read the franles. The students changed their task ^ 
txpm one of reading frames and generating answers t^ one of reading feedback 
and CQpying.it; the presence of feedback directed student attention away ^ . 

from thp relevant' instructional experience. />' ■ j 

By influencing attention, adjunct instructional aid^ can facilitate 

' ' \ ■ 

perfornumce as well. The positive effects of highlighting,- undcrlinLng, , and 
notctatsing on factuar learning are easily handled by DAM, as are the facili- 
tative effects of behavioral objectives on recall ot* obj ecti ve-reiatod infor- 
mation. This, analysis siigget»ts that the effects of higher-order questions on ■ 
learning are not unique; rather higher-order questions are^ simply a type of , 
adjunct H):ist ruct ioua i] a i d. As wi.th all instructional aids, questions whieli 
direct attention to relevant information^ enhanc:.e performance; questions which 
di^rect attontion to irvol'ev'ant information degrade ixvt'fo rmance . Tn the " - . . 
research discussed above, higher-order questions have generally fdcilitatetl / 
posttest performance. The questions have* had' tlii s effect by directing attention 



►'to more information. HhV^^ to more, suflPjects receiving higher 

ortler questions T^^^ more. V * ^ 

shoi^ld be noted that tKe e.ffpcts of instructional aids including ♦ 
questions are not directly causal. QueJ>tions;Vand." other instructional aids, 
serve as one type of input to a. compibx ihformatibn processing system--the 

■ " " ; ■ * ^: * ■• • '-^"V^ - , " 

reading student. DAM suggests- that questions and other adjunct a|,^3 will, 
have maximum effect when , the subj act. perceives his task to ,^e/one of gettin-g . 
through the instructional materia^ls with a minimum of effort. ' ■ Under such, 
conditions, o|uestions influencg the subject by changing what, he pcirceives to 
be the minimum task. If however, the subject perceived -hi^s task Xp be one 
of learrting the maximum amount ^rppi. the material, questions andy other adjunct 
aids nia/ not f a'ci litate his performance. If the subject chooses 'to atpi^nd 
to and learn- all the - information 'the directed attention ejiff^ct of question^ 
wirll not operate. I'or example, Rcy er (1969 Note ^ ) found that'^'cfver-p.TOinptin^ 
in programmed instruction did n6t have dclctrious eFfocts- when the subj qGts..:'' ; 
were highly motivated and attemi^tied'.tp leai'n as much from the i)rogram as they';.,^ 
could. 11us Irinaiysi s suggests that qu6stions will luive a maxinnim effect.o'n . 

■ perib nuance, when the subject is operating under conditions of 4ow- motivation 
or abi 1 ity_(Anderson f, I-aust, 1.973), 

The Nature of Attention in DAM ^ ^ - ' , - ' ^ 

— — — — ■ — ^ , t ^ . 

Ci . • 

The naturp of the attention' refered to Kn the d i rec t od at tent ion model 

shouhl be specified,^ In psychological usage, attention has many refpx'ents ^ 

c.>».» or UM\r..it ioiv:tsO part.^cuiar stimulvif in a ficid, processing of. part LciUs^r 

aspects of stimuli, even concentration and /or arousal , ' In the present • case , 

attention refers to part.iculiir asports of t he rca'di|)>; 1 1 i •> t tuning), process. 

Ktvi.ding (listening) i s conceptua I i zed as a mul t i - st:tged process which involves 

translating visual (aiul i:^ory) s't imu li into ' eiU-o.di ngs of , meanings in tlie work- 

ing memory oi the 'readier. Stage^i in the proct^ss may include: a) recognizing 



the visual patterns of letters and/or words, translating the visual patterns 
to sounds, tAinslating the sound and^r visual pattern^ to meaning represen- 
tations, constructing struotu^fes which encodio,;theAmeanings of sentences, 
paragraphs and^^wnassages, rel-diting new meanings to prior knowledge, etc. 
Psychological contrjayers)? has ra^ed over the precise nature of these stages 
(Gibson 5 Levin, 197^)^ The fine distinctions between the various theorists 
are not relevant hqre/ What is relevant is that giv«n a multistate cojiception 
of the reading (listening) process, we <Ah define attention as used In the' 
directed attention model-, as. dihe processing of presented information as 



or prjesented 

to form a unified mental structure that combines the elements attendedAo. 
In other words, a student' .has attended tb a particular portion of an instruc- 
tional commuiiicat ion when ha has^ encoded some aspect af^ that portion. 

This conception of attention is necessary to account for the effects of 
different levels of qiiestions. To demon^^ate this, compare the Frase (1968) 
and Watts (1973) studies discussed above. Frase found that subj ects -given' 
more general questions di-^ m^re poorly on the posttest. Watts, ().973) demon- 
strated that Frase' s -results wgre produced B^ecause his more general questions 
led students not to attend to the particular associations needed for the 
posttest. Remember th^t the subjects were given short prose paraip |^ which 
'contained information like the names and occupations of men. Iji does not 
seem reasonable to me that 'Watts was saying. that the subjects simply looked 
at only the wor4s in the passage that refered to Occupations when given a 
question like: What were the occupations of the men in the story? The other 
words in the p^assage had .to be looked at, if for no other reason, than simply 
to identify which words were occupation wfl^xds. By attributing the poorer ' 
performance of the general question gr^p to directed attention, ^atts was 
arguing that the' subjects in the genferal question group failed to construct 
a memory representation, pr encoding, which related the men to their occupa- 
tions.' Rather their memory encoding listed the occupations as a separate "chunk. 
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Consi^deration of the results of other studies of instructional aids 
similarly suggests that the effect of the aid is to lead subjects to construct; 
different encodings af the information. A clear demonstration of how differ-^ 
ences in enicoding can influence subsequent test performance is provided by 

\ . ' ■ ■ • : . ' 

Andre and Sola (1976). Andre and Sola had subjects study a list of sentences. 
Subjects received either verbatim or* paraphrased test questions on each 
sentence whi|e studying. When tested with new paraphrase sentences questions 
on a posttestl^ the group receiving paraphrased study*^ questions did better. 

Drawing on results r;eported by Green (1975), Andre and Sola argued that the 

/I ' ' ' " 

group given verbatim questions tried to memorize the sentences as an indepen- 

•1 ■ ' 

dent series of words; they* did riot encode the woi^ds into a Unified cognitive 
structure. Thei groups given paraphrased study questions were led to create 
a unified encoding in order to relate the sentences to the study questions. 

1-.,- ^ 

In contrast, the; Verbatim group apparently memorized the words in the 
sentences as woilds. Construction of a unified encoding increased per^ormaiice 



on the posttest. 

This conception of directed attentidrf and encoding can be made more 
explicit. What I am arguing is that by directilig the reader to particular 
kinds of information, the questions lead the reader to set up a strategy or 

program Tor processing the information in the text. Different questions lead 

^ * . ' * - ,■ 

to different strategies. Figures 1 and 2 illustrate the kinds of strategies 
that might be set bp in response to the Frase (r969) and Watts (1973) type 
general questions. In Figure 1, the general question is: ^ **What are the 
occupations of the men in this story?** This question leads to a reading 
strategy in Which occupation becomes a key or criteria for selecting infor- 
mation for Encoding. As the reader encounters occupational information, he 

takes further action upon it by storing it in an **occupation list.*' Figure 

1 

3 illustrates the kind of encoding* that results from applying the Figure 1 
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Strategy. In Figure 3 the men's occupations are associated to a retrieval 

cue of "occupation of men in story." Of course, this retri^^al cue is 

«' * 
context specific in that applies to occupations read about 'in a particular 

passage in a particular experimental situation. » 

Figure 2 illustrates the effects of th.e Watts (1973) general question: 
"Which man is too old for his job?" In respgnse to this question the reader 
sets up a different set of criteria to'' select information for encoding ati^ 
encoded more, information. Figure 4 illustrates the ki^nd of structure that • 
results. In this second encoding, the retrieval cue is story characters, 
associated to thil cue are the names, of the men and associated to" the names 
^T^the men are occupations and, ages. When the subject is asked to give 
^p^ific information about specific men, we would be" able to do so given 
the second encoding (Figure 4), but not the first (Figure 3) . 

Two qualifications on this description should' be noted. First, the 
probability with which a particular occupation, man's name, or age,' '5tc. is 
encoded is less tHan unity. Thus the reader won't encode or remember all 
the informatioii related to the question. Obviously the reader's posttest 
performance will be reduced by the extent to which his original encoding is 
incomplete. One general task for the kind of model being suggested here is 
to describe what the probabilities of encoding' are under different instruc- 
tional conditions. ' , ^ 

A second ^^related qualif ication_is^that the subject may be' using more than 
one strategy while reading. In response to the inserted question, he may be 
keying on and encoding particular kinds of information; but because of his 
background and his expectations alpi^ whAt information is important and 
likely to be on a posttest, he may also be attempting Jto encode other kinds 
of information, Cert^nly most manure readers assess the importance of 
components of texts they read; the sales of highlighters to college students 
attest to that fact. Moreover, Duell, (1974 j has shown that students do ^ 



remember better what they consider important from a passage. Since DAM 
assumes that only a finite amount of processing capacity is available to 
the reader, the attempt to use multiple strategies will degrajje the success 
of any given strategy and should lower the probability wj^th which informa- 
tion related to a particular strategy is encoded.- . 

Insert Figures 1, 2, 3, 4 about here 



DAM and the Instructional Value of Studies Assessing only Verbatinr Recall 

If DAM provides a full explanation of the level of question -effect, 
then question level does not seem to be -an. instructionally interesting 
variable. It is hardly surprising that information learners pay attention 
to (encode) is learned and Stained better than information they do not 
attend to (encode). If a higher level question merely directed the reader's 
attention to more of the passage, it would be possible to simulate its effect 
by asking- a greater number of lower level questions. Most educators and 
theorists who have dealt with levels of questions would not be comfortable 
with that notion. The general belief has been that higher order questions 
exert a qualitative, not quantitative effect. It has been believed that higher 
order questions lead the learner to acquire something else in addition to 
simply more. 

Unfortunately the designs of studies discussed above have precluded the 
demonstration of anything except quantitative effects. These studies clearly 
fail to distinguish between the learning of a prose passage and learning 
from a passage. In laboratory verbal learning research, subjects are; typically 
asked to learn arbitrary lists of nonsense or real words. In such experiments, 
the list represents the universe of to-be- learned material , Most prose 
research has treated prose passages in the same way. Researchers have treated 
prose passages as complex word li'sts; the passage and its words have become 
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the material to be learned. The Rickards and DiVesta (1974) study provides 

» . • ■ • 1." 

a typical examp^le. After ingeniously varying the type of adjunct question, 

,', i ' , " ■ 

' . • •> 

Rickards and DiVesta simply required verbatim recall of the passage. This 
type of testing procedure makes it impossible to demonstrate anything other 
than a quantitative^ efffect ^.or higher level questions. When a verbatim ^ 
recall or recognition procedure is, used on the posttest, only the quantity 
of verbatim recall can be assessed. v , 

While, such recall procedures have some theoretical and pedagogical 
Tinterest, they also represent a clear misunderstanding off the instructional 
use of prose . When a teacher assigns a passage he seldom regards learning 
6f the passage as the end of instruction. Rather the teacher expects the 
jpassage will communicate important ideas that' students will employ in the 
future. Such ideas may include concepts, principles, skills, or problem- 
solving techniques, etc. Regardless of the specific nature of the ideas, 
educators want students to le^i ^from passages, not to memorize passages. 
From the educatoj^'s perspective, a prose passage is a device for altering 
the reader's knowledge of the world. Different levels of adjunct questions 
may facilitate the ease with which such alterations occur, but studies 
which require only verbatim recall will never reveal these effects. Such 
effects become apparent only when the reader must demonstrate his knowledge 
in a transfer situation (Anderson, 1972). , 

■ 

Studies Assessing More Than Verbatim Recall 
Fortunately, a few studies have investigated the role of adjunct questions' 
in producing transferable knowledge. Although only a handful in number, these 
studies demonstrate an effect of higher level questions which DAM cannot 

ft 

handle. , 

Anderson (1972) had argued that paraphrased questions induced a higher 
level of processing than their verbatim counterparts. Anderson and Biddle (1975) 
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attempted to^sHow that/paraphrased questions would enhance learning from 

a prose passage. » Over a series of 4 studies they were unable to show any 
t. • 

convincing direct facilitative ^fect ot such questions. Andre and Sola 
(1976) criticized Anderson and Biddle's work on two grounds. Anderson 
and Biddle's paraphrased adjunct questions were given onl}^^fter subjects • 
had read the passage. Used in this way the questions could not induce 
subjects to process the information more deeply. In addition, Anderson 
and Biddle's post test was composed Bf questions used in instruction and, 
as such, did not provide a pure test of their hypothesis. As noted above, 
Andre and Sola (1-976) ^'^^^ able to demonstrate that when adjunct paraphrased 
questions could guide semantic encoding of the presented information and when 
the posttest measured such semantic encoding of the information, the para- 
phrased adjunct questions led to greater performance. 

Hunkins (1969) had 6th grnrl?^ students study over a four week period 
social study materials contaiiiipr either ^'knowledge'* level or '^evaluative" 
level questions. At the end of the period they took a posttest containing 
questions at' all six of the levels in the Bloom taxonomy. The only signi- 
ficant main' effect for treatment questions occurred for evaluative level 
questions--students who received- evaluative level questions during instruc- 
tion did better on new evaluative questions on the posttest. Type of adjunct 
question also entered into two three way interactions with sex and reading 
level. Unfortunately, it is difficult to interpret the Hunkins* results 
as the paucity of methodological details in his report makes it difficult 
to know exactly what was done. For example, it is not clear how long the 



experimental materials were, how many adjunct questTons" were used, whether 

the same number of factual and adjunct questions were given to the two treatment 

.■ ■ ' 

questions, whether the adjunct questions were inserted in the text near the 

material that provided answers for them or massed at the end of the sections. 
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and whether students received feedback on the questions. In addition, the 
materials were used in on'going classrooms, but the relationship between the 
experimental mi^terials and the ongoing instruction was not made clear. More- 
over, Hunkins repprts initial interrater reliabilities for the assignment 
of adjunct questions to categori^. These are quite low (mean =? 52.2%) 
suggesting that the treatments were not reliably different. Finally, the 
data appear to have been, analyzed incorrectly. Hunkins assigned intkct 
classes to treatments, but used individuals as the unit of analysis. Hi 
the proper hierarchical analysis been performed, it is unlikely any effects 
would have been significant. Given these problems, it is best to regard 
the Hunk^s results as providing only weak support for a level of questions 
effect on higher-order learning. However, Hunkins is to be commended for 
attempting to assess 'more tKan simple factual learning. 

Perhaps *th^ most intriguing study to examine higher order questions is 

: ^- ' . z' 

that of Watts'fand Anderson (1971). Watts and Anderson wrote passages which 
consisted of.brifef descriptions of psychological concepts and principles. 
Each of five pas$ages contained, (1) an example of a psychological concept 
or principle, (2) a verbal statement of the- concept or' principle, (3) the ^ 
name of a psychologist associated with. the principle, and (4) a second 
example. Five questions were written for each passage/ The Name questions 
asked the reader to identify the psychologist associated with the principle. 
Each of two Repeated Example questions asked the student to recognize one or 
the other of the given examples as an instance of the principle. Two 
Application questions asked readers to recognize a new, unfamiliar instance' 
of the concept. Each reader read all five passages. , After each passage ^ V 
appeared one of the five questions. Thus, there were five groups and 'Subj ect-s-- 
in each group answered either name, one or the other of the two repeated 
\ examples, or one or the other of the t^o application adjunct questions while 

studying the material. Subsequent to reading the passages, each student 

" ^ ■■ ■ - ■ . / . ' ' . / ^ ■ 

V . ■../■■■ 
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tompleted a test consisting of all 25 questions. The important result was 
that, the groups asked Applicdtion questions during reading performed much 
better on new Application questions and jibdut^ as well on the other typds of 
questions as did the Name and Repeated Example groups. That is, the 
application questions groups were better able to transfer their knowledge of' 
the concepts and principles to the new examples, , 

A study by Dapra and Felker (Note 2) supported Watts and Anderson's (1971) 
findings, Dapra and Felker had subjects study materials on basic conditioning 
concepts and principles. Subjects answered either comprehension or verbatim 
questions while studying. Verbatim questions demanded memory of the exact 
words in the text; comprehension questions required the subject to understand 
a paraphrased version of the message presentciti,, in the text or to apply the 
concept or principle,. These distinctions; were based on the AnderscJn (1972) 
pap[|^r. Subjects were given two posttests in the following ord*: a problem- 
solving, test containing descriptions of situations' to which conditioning^ 
principles could be applied and a multiple-choice. posttest containing new ^ 
comprehension items followed by verbatim items. The readers given compre- 
hension adjunct questions scored higher on the problem solving test, but 
did not score higher on the new comprehension multiple-choice items, 

Anderson and Kulhavy (1972) presented subjects with the definitions of 
concepts with which the subjects were unfamiliar. While studying the concepts, 
subjects either tried to make up a sentence using the defined Concept (word) 
or repeated the definit^ion aloud three items. On a posttest that asked subjects 
to recognize new instances of concept, subjects who used the concept in a sen- 
tence performed Ibetter ' than subjects who repeated the concept aloud, Anderson ' 
and Kulhavy argued that using the concepts required the subjects to, semanti- 
cally encode the definition and hence be able to use it. 

In a long-term study, McKenzie (1972) had eighth grade stud5nts take 

A 

weekly quizzes that required either recall of given facts or drawing inferences 
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about the political interests of groups discussed in the material. After 
^ight weeks subjects took a posttest which required recall of facts, new 
inferences about the groups considered in th^ weekly quizzes, and new infie'r- 
ences about groups not previously considered. Type of quiz did not influence 
either recall of facts or new inferences about new groups, but did influence 
performance on the new inferences ab^out the old groups. Subjects given 
.'.inferential quizzes did better than subjects given factual quizzes. 

Moore (Note 3) ^reported apparently disconf irminjg results. Subjects in 
Mftofe's study were given either verbatim, paraphrased, or application adjunct 

«stions while studying. a passage. Control grdups received no questions, 
and eight days after reading the passage the subjects took tests on the 
material. No significant difference between groups were found. However, 
Moore's stiidy was designed to test the general facilitatOry effects of 
inserted questions predicted from his interpretation of Rothkopf^s (1963, 
1966) mathemagenic notions. For this reason none of the posttest questions 
were^directly or indirectly related to the adjunct questions. Most impor- 
tantly in the present context, none of the posttest questions asked students 
* .... ■* , . ' 

to apply the concepts and principles that had been involved in the adjunct 

questions. Moore's data, therefore demonstrate the effect of adjunct applica- 
tion questions is specific to the content with which they are concerned. As 
Moore (Note 3) noted, such effects are still educationally important if they 
facilitate learning of specific concepts and principles. ^ 

McConkie, Rayner, and Wilson (19737 attempted to use different levels 
of. questions to alter the reading strategies of subjects. Subjects read 
six passages on^diverse topics and. received 1 of 5 types of questions while ( 
reading. The 5 question types were: (1) factual questions with word answers, 
(2) factual questions with number answers, (3) structure questions , which 
involved telling the^order of things in the passage or the amount of passage 
devoted to subtopics, (4) far- inference questions, which asked for things 



as the best title for a passage or the authors purpose in writing the passage, 
and (5) recognition questions, which asked students to pick out words and 
^phrases from the article from among a list of distrators.. Two additionaT ,f. 
• groups received/ one of each type of ques^tion after each passage or two par- 
ticular- inference questions after each passage. Except for the latter group, ' 
each group received 5 questions aft^r each passage. After cdmpi^eting the 
passage, subjects took a retention test whiqh contained factual word, factual 
number, structure, far inference, "and recognition items. Wheth^er these " 
^items were the same j.^ those u-sed in the., .passage was nofmade clear, although^ 
McConkie e^ al_ indirectly sug'^est^that the posttest contained both new and 
-^repeated items. Type of adjunct question influenced both reading spee^ and 
posttest performance. As McConkie et^ al_ suggest the differences in posttest 
performance are not easily interpretable. The far inference adjunct questions, 
which come closest to the kind of question we have been'calling higher order, 
» did not facilitate performance on far inference posttest questions. Since 
McConkie et^ aj^ imply that the posttest questions and the adjunct questions 
referenced at least partially different information, this result may support 
the Moore (Note 3) findings. ' 

^ Part of the problem in interpreting McCdnkie's et al^'s results is thfe 
ill-defined .nature of the higher-order questions. I don^t think it would be 
possible to even quasi-replicate this study from the information given in the' 
report. One simplly could not draw, up sets of structure arid far inference 
questions that would match the characteristics of McConkie 's et> al 's questions. 
A second problem is that posttest data were analyzed, u^ing a repeated measures 
ANOVA in which posttest question-type is a within-subjects factor . They report 
a sijgnificant interaction of Adjunct question type and Posttest question type; 
but do not report the simple, main effects of Adjunct question at e^h level 
of Posttest question type. But it is the latter effects that are- of n^ajor 
interest. From an instructional viewpoint, it would be important to know if 



different typ^s of adjunct Questions make differences in instructionally 

meaningful, kinds of outcome measu;:es. The oy^all , significant interaction . 

merely indicates that there may be difference?' of interest within the dAtia 

but does not isolate them. In the case wherO:.^psttests contaii/in 

• • ' . • ■ • 

i::ionally^ meaningful distinct types of questions , s'epardte between subjects 

ANOyAS;^ On each question type would probably be more informative about the 

relationship in the dat&. If the researcher deems, it necessary to use an 

overall analysis, an appropriate MANOVA which would yield univariate Fs 

would'vbe a better choice than a repeated measures ANOVA. Doing the latter 

analysis without providing follow-up tests .will typically produce only 

confusion about the data. ^ ) ,. 

Shavelsbh, Berliner, Ravitch, and Loeding (1974) compalred the effects ^ " 

r. ' ' '. . . 

of higher or lower order questions! given before or after the passages that 

* ' ' ' " . ■*■ . ■^^^'^ ^ \ 

answered them. A control group receivi&d no questions. Higher order questions 
■ . . ' ; • ' ^ ' ' ' . ■ -^^ ■ ^' 

placed" aft^ the releva^lt material tended to produce better performance than 

did other questibfi-position combinations. '^However the no-question control 

group did about as well as t^ie higher-jorder q^uestions group . differences 

between oonditigni^ were not generally • significant. A number of problems 

make clear interpretation Of^this study difficult. The authors indicate that 

the low-level questions w^re at the knowledge level in Bloon\-;et al Taxonomy 

and ^hat the higher-order questions; were at the comprehension, application, 

and analysis levels of the Taxonomy. However, the^xamples provided in their 

article were not obviou^^ly at different -levels and th^ specific relationships 

between the text and the adjunct questions Were not made explicit'. For 

example, the authorsdid not indicate if the knpwledge level questions were 

verbatim repetitions of text or involved paraphrasing. More importantly, 

the relationship between the higher order adjunct questions and the higher 

order transfer questions on the posttest was not made clear. Whether these 

teste^d the same or different ^concepts or principles was never made clear. 



?bfn* the*' article, I infer that different conceptijfcnd principles were tested, 
^xjt this is only a guess. If the concepts and principles tjpsted were 'differ- 
ent, then the results support the Moore (1975) study discussed above. 

The results these studies suggest that when students are;, given 
adjunct application questions about concepts and principles, as compared to 
adjunct" factual questions, their ability to use knowledge of ;f:b^ concepts, 
and principles to recognize new examples or solyfe problems involving' the con- 
cepts and principles is enhanced (Wat'ts Q'^Andfersbn , ^971 ; Dapra 5 Felker,. 
Note 23 . the e.ffeCts pjf' the questions appear to ''b^ specific to the concepts 
and principles asked abput in the adjunct' questions; the acquisition of other 
concepts and principles discussed is not faciliated (Moore, 1975; Shavelson, 
et al_, 1974; McConkie er al^, 1973). The effects of other types of higher * 
level questions are much ies5 clear. ^ ' 

Some recent results originating in my laboratory raise quest ions^ about 
these generalizations. (Andre^ 197^, N^t^e. 1 ): In a series . of t^iree -Situdies 
based upon the Watts and Anderson paradigm, I -asked subjbtts to -read prose 

pafssage^ explaining psychological concepts and principles and to answer ' . 

f. •• . • . ■ > ■ ' , - - * 

either factual' or. application Questions whi le reading. The materials were 
the Watts and Anderson (1971) passages pljUS : gidditional passages , con- 
structed to match the or iginal: ^materials ^n^^iiiiyle and format. Questions were 
' were placed either before or after the rel-^a^t parts of the, passage j> hence 
. the basic design was a 2 (Question Position) X' 2 (Question Level) factorial. 

Over the three st-udies, and considering only the new application items on 
< the posttest, the, results were- as follows. In Experiment 1, College student 
subjects given factu^l^ questions befb^fe'the relevant -pprtions of the passage 
did best on answering new application questions . In Experiment 2, again 
using college students, no significant differences ' found ^' .In vfe 
3, high school students given factual questions ther .fo«re or after the 
relevant passage portions did significantly better on new application items 
than did students given application adjunct questions. : 



These results are ijicbnsistpnt with each othisir and with the Watts and 
Anderson findings'. This inconsistency raises ^^efetions about ,;tlfte 'generaliza- 
bility of the effects of adjunct alpplicaion que.stions. The iri^sults suggest 
that the effects of questions are, moderated by other variables, 

. One possible such variable may be difficulty level o^ the adjunct ques- 
^tions . The absolute/ level pf performance on t;he adjunct questions was poorer 
in , the Andre studies than in the Watts arid Anderson study. It is reasonable 
to suppose that if the students are unable to answer the adjunct questions, 
the; questions, will not facilitate their performance. In fact, if the adjunct 
questions prove particularly difficult they might reduce overall performance 
by increasing the students friistration and lowering his motivation. Another 
variable may be the ability level of the students- Shavelson et al^ (1974) 
reported^-'-'that higher- level questions had less of an l^^ffQct on higher-ability. 
Students. (Although this patterri' of interaction does not seem'to b6^con- 
firmed in* the Hunkip's 1969 data.) 'The college students used in the Andre 
Note 1 research Would represent a population selected for ability; the: high ^ 
school students came from a high school whose ;s*tudents ' mean ability leVieils 

, \ ' ■ ' ■ ' * .'^ ■ 

average^above national norms. Clearly such speculations cannot account /for 
the differences between the Andre Note 1 and Wat|||||and Anderson. ,(J971) result 
However the speculations do suggest that future research on question level - > 
shotJld-ekaipine possiblV^ trait-treatment interactions. The inconsistencies in 
the extant research underscore the3eed for subsequent research-^ 

The importance of the'^incOnsistencies in this research should not be 

overrated. . Only a very few studies hiave been performed and only a very 1 

• ■ ■ ' ' . ■'. ' * , ■ '■ . 

■ limited number of variables con^^idered. Given the wide differences in method 
atid procedure across studies, such inconsistencies should not bte entiri^y 
unexpected.; If, as I*belieye, questions exert only an indirect influence on 
the. J.earner,^^^^^^t^^^ different effects on different, 

learners -^iW d^^^^^ situ^Jioris. . The effedtis of questiphsrw^ be determine 



by the total input influencing the learner, the-prior habitfs and reading 

' ■ ""t^ , , . " > • 

f * 

i^trategies of the learner^ ^"^-^^Ji thC'learnpr '4;qmbines wd transforms this . 

info^ati-on irttova perception of hi^, t^ask and goals. OtlTy'^when the con- 

figuration of the system Is'^ sucih '^tiSiit' the questions lead the learner tQ ^ 

process the materials inwaysb© Would not otherwise- have done will questions 

influence learning and retention, and ti^dti'sfer. , The effects of questions will 

be itdnditiona^^V^^ aspects- of .the learner's processing system. This 

view of the' learner ^nd of the influence of questions on/ the leatner is 

pursued in the section below which proposes a model of learner and his com^^^ 

prehension/retention/ transfer processes, - - ' ' ^ 

' • , . J ' , • • • ... 

Higher-Order Off ects from- Higher Level Questions 



It doei> seem clear that under apjii-opriate, but as yet not fully specifie 
conditions application questions produce bptt.bt^'^ situations/ 
As developed above, KAM idoes not seem adequate to account f*t>r this effect. 
Under PAN1;. higher ofder questions had their effect by leading su>j^cts to;:fv^";' 
semantically • encpde more of .1^^^^^ in a passage. If DAM wer*5 suffi- ^ 

cient ;to account for the transfer effects that haye been obs.eyved, then those 
effects could be simulated by Bsking subjectjs several low order questions 

. ■ , , , 9 

' • . ' ' • -i • 

^about the passage instead of asking one highfer prd^^ the face 

of it, it appears difficult' to account for^the o.b transfer effects in ' 

this way.. ^In ,the Wartts and Anderson (1971) $tudy, for ^example, the .higher .. 
order application questions required that subjects make use of the preseirt«5d^^^^ 
information in a new way. The* questions required that subjects go ^^beyond/ 
the informaHon "given**, i^-'Brilner/s fortunate phrase. How''such.;an afi"ect 
could be duplicated by asking several factual questions about the passage "'is ? 
not clear. Thus the ^^M model is not sufficient to . account for such higher 

' ' , < vv ;.:v,; . A 

order or transfer effects of inserted higher-ot^der questions. A model which 
is' designed to. account j^f or such effects *is (^escribed in the final section ., 
beltfw, . BeiFore turning to. that description, it is approrpriate to note some 
general problems with the existing'^ research on levels of qu^^^ 




Some Methodological Criticisms '^^^^ ^ 

One major problem wit)v#esearch on question level lies in the specifica- 
tion of the levels of quest\^i^in_ th^studiesT • Anderson (1972) criticized 
educational researchers and journal editors for printing rese^r^ch reports' 
which were not replicable because essential features of the materials were 
not fully described. The problem still exists. In several of the studies 
reviewed herein, essential information such as whether the adjunct questions 
were verbatim or paraphrased, whether the adjunct questions were repeated 
on the- po?bt^t, and whether the posttest tested' over concepts and principles 
also tested in the- adj unct questions or on additional concepts and principles 
was simply not given (e.g., Al.len. 1970; McConkie, 193^; Shavelson, et al_. , 
1974). Part of the reason for this ambiguity has .bpen the lack of an agreed 

* 

upon system for classifying questions. However, thi lack of a system should 
imply that researchers should be especially detailed in describing the pro- 
cedures by which questions of varioQ^ levels were developed, because a reader 
cannot refer back to a system for objectively generating similar^tiuestions . 

A criticism is that many of the studies failed to provide. data 

on the subjects • performance on the adjunct questions. Significant e5cceptions 
are the- Anderson and Biddle (197S), Watts and Anderson (1971), and Andre aiKl 
Sola (1976) studios. Holland (1965) denwnstrated that answering relatively 
easy and relevant tiuestions during instriKtion coultl facislitate performance 
but that answering difficult but s1;ill relevant questions would not. Where 
data is available, it is clear that higher and lower level adjunct questions 
do differ on il i f f iculty .. Thus, in the absence of data on the subjects per- 
formance on the adjunct iiuestions, interpretations of stuilios varying level 
o (" tfuest ions heconjes exceedingly teiuious. ' 

A third problem, also notetl above, deals with the nature of output or 
posttest mea.stues. Instructional communications could have a inimbor of 
potential outcomes; knowledge ^^aincd from such communications could h't^ 



ass^ed in a variety of ways. It .seems appropriate that if levels of adjunct 
questions are to be varied in instruction then various levels of questions 
should also be assessed on the posttest. Studies which rely only on one 
output measure, such as factual recall, reveal too little. Virtually all 
question level research should use multiple output measures. 

Finally, it sedms appropriate tha*t investigators interested in the 
instructional effects of different levels of questions should devote more 
interest to the individual characteristic of the students. Under the model 
of the learner informally presented thus far, questions influence the learner 
by changin^j his perception of the task and the strategies he uses to pursue 
the perceived task. This conception suggests thatvcharacteristics of sub- 
jects that are related to how they perceive and act in prose learning (read- 
ing) situations will also interact with questions. Some evidence for such 
interactions was provided by Shavelson, et al. (1974). Like most^experimenta- 
lists, prose learning researchers have for the most part ignored individual 
difference variables. The inconsistency in results noted herein suggests that 
researchers do so at their peril. 

Most of these criticisms relate to the nature of scientific research 
and how such research should be reported and described. Good research and 
good science begin in careful observation and description. All of the soph- 

■ ^i^b ^ . , . 

isticcited techniques of modern educational research :are useless without these 

characteristics. Careful observation requires that" the conditions of a study 

bo noted as fully as possible. Such conditions include the nature of the 

suhjects. and the nature of the experimental material>^. It behooves researcher 

to find out us much as they can about subjects, materials, and so foVth. As 

1 * ■ 

an 'example, how many' prose researchers know how many males and-f elmil es parti- 

>♦ 

cipated in their last study or what the reading difficulty level of the 
material was? Careful description requires that sufh desc r ip t i ve ' data UhonX 
(h<- coiuiitiuns of the study" be reported. Unless such data, is reported it is 



. almost useless to .report the article. Journal editors and reviewers bear 
a major portion of the responsibility for insuring that such information / 
about research is available. ' ''•}■ 

The point is not that such data is necessary for understanding a 
particular study in isolation. I agree that such information inay b^ irrle- 
vant to the hypothesis of a particular study. In fact , the; hypotheses of 
an experimental study usually can be tested without knowing such information. 
Rather the. point is that to intelligently relate , compare , and integrate dif- 
ferent studies investigating similar but different hypotheses with different 

* subjects and materials, such descriptive data^ is essential. In general, 
educational and psychological, researchers in the prose learning area have 
done a bad job of reporting such information. Certainly integration of the 
level-of-question research would be facilitated if such information were 
known . 

Towards a Model of Productive Learning From Prose 
The purpose of this section is to present a model of productive learning 
The model seeks to organize what is known about question level effects and to 
describe how information presented in prose is assimilated and stored in 
cognitive structure. In its current form, the model is primitive and some- 
what speculative. I envision it as a guide for future thinking and research 
rather than a complete fornuil description. Research on prose learning and 
the use of ciuestions in prose has been mostly empirical and noh- theoretical 
in nature. One value of the proposed model is that it can serve as a theore- 
vtical guide for future .research in this area; -The model is certainly not a 
uni(iue contribution on my part as it draws upon ideas and representations 
from a variety of sources. These contributions are noted below. Like most 
current models of ^memory, the present model views the learner as a complex 
information processing system. The basic structure of the model is illustrat 
in ri^'.urt; S. 'I'lio now familiar toatures of .sucli models: .';ensory ro^; l.stor.s , 



short term memory, an executive, long term memory, etc. are present. These 
structural features are highly interrelated, connections are shown between 

long term memory and sensory- register since schemata store in long term 

' '' • ■ ' ■' ■ t 

memory may. be assessed and used in the pattern-recognition procedure. The ' 

short term memory is conceptualized somewhat differently than in the original 
^Waugh and Norman , (1965) and Atkinson a^^Shiffrin (1968) models. The short 
term store is considered to be the place where currently thought about informa- 
tion is stored and limits are not placed, as in the Jamesian notion of conscious- 
ness (James, 1892; see also Shiffrin 6 Schneider, 1977), upon the nature or • 
type o^f information in STM. 

The model postulates that the human infoxmation processing system con- 
tinually receives information from sensory receptors and carries out various 
pro'^rams of processing upon such information. The way in which such information 
is processed is determined by both the nature of the incoming information and 

the. context in which such information is received. Context is used here to 

■) , — ' 

designate the other information received along with a specific item of infor- 
mation and/or temporally or spatially contiguous or adjacent with such informa- 
tion or information retrieved from memory as a result of such information. 
Such other information can include sensory information received slightly 
before the specific item of information to be processed, internal information 
activated in response to incoming information, internal information related 
to Intent Ions, beliefs, attitudes, and goals of the information-processer . 

A specific example involving reading may clarify this notion. Consider 
a case where I am reading a section of an encycIo{)edia In order to obtain 
the scientific name of a certain animal. I look up ^'??el ection using a common 
name T believe correct, but find that I have an article on a related animal. 
In readLuK this article to discover this, 1 find that the article presents 
Vhe comnK>n nai\u^ o f the^ an Lma I - i t concerns in the f irst sentence. The second 
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sentence contains the scientific name* Other sentences in the article 
describe the animal and also include references to other related animals. 
In reading these subsequent sentences in the first article, I find I have 
the wrong article and turn to a second aTticle, in the encyclopedia. While 
reading this second article, 1, as an information processer, ' have several . 
complexes of information influencing my interpretation of what I am reading. 
To illustrate, I am influ^rjced by (1) my intention to find a scientific 
name, (2) my previous knowledgi^ of scientific names such that they are often 
Latin and italicized, (3) the information abbut the order of information in 
the initially read article; I probably expect the scientific name to follow 
the common name, (4) my uncertainty that the common name I have looked up is 
correct. In reading the article I will probably scan to find the latinizeci 
italized scientific name, and further scan to verify that the animal I am 
reading about is the animal I want. Were any of the above items of informa- 
tion to change, my processing of the article would change. 

The information processing . system is further conceptualized as containing 
at least two distinct memory stores, an episodic memory and a so-called 
semantic memory (Tulving, 1972). ' As Tulving has argued, the episodic memory 
contains memories for the personal stream' of events encountered by the infor- 
mation-processei^. These memories are organized spatially/temporally; that is 
they are tied to places and sequences. Such events are not represented in 
this memory in the form^in which they were inptit, rather episodic memory con- 
tains a record of events the system has interpreted (encoded in current 
cognitive jargon). ' ' ■ 

I. ^ ' 

Like Paivio, *I believe that the system can represent "events either imagin 
ally, tliat is through the visual information processing system, or verbally 
through the auditory information processing system (Paiviu, 1969, .1971, 1974). 
(In fact I would probably go further and suggest that the rei)resentat ion may 
ho In terms of any sensory feature depending upon the original interpretation 
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by the system, l>1it. this is not crucial to the development of the model,) 
...The not:^ that episodic memory contains st®re.d interpretations is not new 
and in a sense represents one problem for the model. One task for model 
is\.tp describe just when incoming information is interpreted enough to be 
stored in episodic memory. What I think happens is that the system stores 
information that it has made sense of (and perhaps the notation that some ^ 
information was not interpretable) . Of course this belief means that I 
must describe precisely what "making, sense" is in terms of the model. Since 
this problem is not central in the present context of developing an account 
of learning through reading, I am going to say that episodic memory contains 
a record of stored perceptions arid ten^orarily leave the problem of perception 
in the hands, of theorists whose central interest it is ('see for example, 
Lindsay and Norman (1973). Considerable progress towards an information- 
processing account of perception has been made.) ^ 

Semantic memory contains the system's abstracted or general£ze<5 knowledge 
The contents of semantic memory are concepts, principles, rules, skills, etc. 
that are broader 'than specific episodes. Under the present model semantic 
memory is broader than what many theorists have^_hel^d^to be semantic memory. 
It is the repository of the knowledge that makes comprehension of input, not 
simply linguistic input, possible. This cojiception is congruent with recent 
descriptions of semantic memory (Anderson, 1977, Rumelhart t\ Ontony, 1977). 

Specifying the nature of the content of semantic memory has been a pro.- 
blem to which cognitive psychology has devoted considerable attention. A 
number of different models of semantic memory have been proposed (Anderson 
l\ Uowor, 1973; 'Collins T, Loftus, 1975; Collins t\ QuiHian, 1972; Kintsch, 
1972; Rumelhart, et_al^l^>72; Rumelhart Ortony, NotolO^.in goncrai, such 
models represented semantic memory as a network of interconnected ideas. 
Networks contain nodes and connections botwoen nodes; l-igures 3 and 4 contain 
visual reprosontatlon of networks. One major set of differences botweiMi 



proposed models of semantic memory seem to be primarily in the naturefSr the 
'elements' contained at a node and the\ nature of/ the connections between nodes 
A second controversial area involves the extent to which concepts are hier- 
archically organized and ri^dundancy of information associated with related 
nodes is permitted. Since this second controversy is not essefitial to the 
current presentation, it is sidestepped here. ^ 

•The question of how knowledge is represented in semantic memory (What 
is at a node?) is very important for understanding the effects of higher-level 
questions. In my estimation, questions influence the nature of the repre- 
sentdtion formed when subjects acquire new information in semantic memory. 
Previous models have presented two general means of representing knowledge 
within semantic, memory. The most common scheme is to have each node-repre- 
sent a concept to which are associated various features (other concepts). 
The connections between a given concept and other concepts are of different 
types called 'labeled directed relations.*' Examples of systems possessing 
this type of organization' are Rumelhart, et_ aj^, 1972; Kintsch, 1972; Collins 
and Quillian, 1972; and Smith, et_ al. , 1974. 

Within this type of feature system one of the debatable issues is whether 
the features art^ entirely linguistic or whether they can include sensory- . 
imaginal information. Anderson (1975), Anderson and Ortony (1975) and 
Walker' (197 5) have argued for the necessity of sensory information being 
available to semantic memory. The present model similarly adopts that posi- 
tion. To handle sensory information the present model argues (similarly 
to Lockhart , 6t al_, 197^) that there can \ic connect ions between semantic 
mcraory aiul e])isodic memory. In the ]irescnt model, sensory information is 
represented as beinK stored as episodes in episodic memory. As part of the 
|)rocess of acquiring new concepts connections are established between , 
episodes and the labels given them. The labels form an entry to semantic 
memory. In this way certain kinds of sonsoiy information can be available ' 
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or a<;cessed from semantic memory. How and when this informat^ioii is accessed 
or used ts discussed below. . . 

The other general mechanism that has been used to represent a cohcept 
in semantic memory is the schema (Anderson, 1976; Rumelhart 5 Ortony, 1976; 
Schank, 1975). As used by various authors, a schema represents a'^plot'^ which 
contains variables. The plot represents the constant features of a particular 
concept while the variables represent the changing aspecip* In recognizing 
a concept the variables are instantiated. The schema contai.ns informati'on 
about the typical values that variables in the schjema may take. These typical 
valuer may be used in the schema when the actual values are'unknown. In 
computer terminology a schema is a kind of catalogued procedure. 

To get an idea of how a schema operates to produce understanding consider 
the following story: The man entered the door. He exami ned the loaves and 
selected one. He realized he had left his, change, so he returned and got it. 
At a surface level the story is disconnected and the sentences are not 
necessarily related to each other. Yet most readers quickly recognize the 
story as one about a man buying bread in a grocery story. In comprehending 
the story it is likely that readtjSl fill in details like shelves, cash ' 
registers and clerks. In the view of 'sCliema the-orists, readers have a^'buying*' 
schema which coordinate the separate information from the sentences of the 
story and also supplies the mis^^ng detlii Is . The schema supplies a theme 
that makes sense of thef stimulus story. 

The importance of such integrative schemata is demonstrated by 
Dooling and Lachman's (1971) research on ambiguous stories. Subjects ■ gi^vcn - 
the theme recall the stories better. As an interesting demonstration that 
subjects do search for integrative schemata, I have used tlie Dooling and ^ 
Lachnian stories, which are very ambiguous, as class demonstrations. Very 
olten at least one or two students (out of ISO to 200) generate the a[ipro- 
priate theme on their own. 
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It is this sense of schema that Bartlett (1932) and other classical 
schemata theorists were describing. This conception of schema also is 
congryent with Neisser's notion -of reconstructing dinosaurs (Neisser, 1967, 
p. 285). The example also illustrates what current theorists mean by 
schemata (Anderson, 1977; Rumelhart and Ortohy, .1977; Bobrow and Norman. 

1975). ^ : 

A major difference between the associated feature and schema appfoache 
has been the extent to which the concept possesses, particular defining fea- 
tures,. Under the feature view, the concept is believed to have a set of • 
features associated with it that are true of every instance and are used j . 
to determine if a particular instance is an instance of the cateEory ' 
These features can be called defining features (Smith, et_ al , 1974).; The 
contrasting schema position is that concepts do not have defining features, 
rather instances of concepts possess a family relationship. Anderson 

describes this family relationship by saying that concepts possess only 
characteristics fea^tures, not defining ones (Anderson, et al . 1976, p. 668) 
This latter conception of how a concept is represented in semantic memory 
bears certain similarities to the concept of a fuzzy set (Zadeh, et al , 
1975). ' ■ ' ■■ 

In the present author's estimation, there is no real -conflict between 
these po'^itions. I argue below that the nature of the ' representation' of , a 
concept in memory. is determined by the kinds of tusks that the subject is 
called upon to perform with that concept.,, Certain kinds of^asks lead to 
the development of either relational features to other concepts or to the 
development of catalogued procedures. A particular concept, principle, 
idea niay h;i;ve multiple representations depending upon^the subjects past 
experiences with this concept. The present model adopts the general net- 
work nepresentiition adopted by most authors. This type of lepre.sentat ion 
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is almost necessitated by the obvious facts of the interconnectednes3 of 
ideas. E^ch node in the network represejj;^ a concept, principle; or skill 
or other idea. Each node -in the network can be connected through labeled 
associations to other nodes, to instances of the node in episodic memory, 
or to catalogued procedures that represent particular processing operations 
that may be carried out on the , information^T.elated to the node. In 
addition the system is .assumed to possess some general reasoning capabilities 

that can be applied to information connected wi^ a node. Whether a partiGu- 

[ ^ ^ ■ 

lar node contains any given kind of connection will be determined by the 

^ ' % • ■ ' 

kinds of tasks the learner has performed in the development of that node.. 

Thus semantic memory is a kind of historical record of processing the sys~ 

tem has carried out. If the learner has been a,sked .to relate a concept to 

other concepts or \to« produce verbalizations o^ defining characteristic^, tlien 

fthe representation in semantic memory will contain those type of features*.' 

If the learner has been asJCed to recognize unfamiliar instances of concepts, 

then he will have a schema (catalogued procedure) ^for doing so associated 

with the concept node. If the learner is called upon' to perform a task with 

a particular node and the necessary information is not ^tvM^Lable in semantic 

memory, then the sys'tejjn will apply its general r^soi^ing powers to the 

episodes dissociated with the concept and will attemjVt tp carr]!'' out the task; 

•. . ' ■ ' ' ' . ■ • . f' ■' 

■ ^ ■■ ... ^. ■ .. - f ■ • 

Basically this model argucs^hat the contents of .splnantic memory for a'. 

... ^ - . ^. I ^ 

particular ' node contain the results, of the various- kinds of processing opera- 

/ / 

tions that have been conducted with that node. T^hese contents o^f semantic 

r 

,. • . ■ . ■ ; - ■ V ■ ■ 

.memory partially determine the kinds of tasks the subject can do with the 

nodo; but the subject is also able to fall baqt on general reasoning powers 

. •■ , ./ . .. '/ ■ ■ ■ ■ • 

to attempt new tasks. / 

To make this more explicit, let us tr^yce through the development of a 
particular node in a semantic memory. Lot lis choose the [)rinciple ot 
intermittent reinforcement as the to-be- learned information. Prior to 
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acquiring this principle^, the, learnei: has informartion about prerequisite 
concepts stored, in memory. He knows what reinforcement is, what intermittent 
means, v^haV extinction is ete. What he has to acquire is (1) the verbaliza- 
tion the intermitt^ent reinforcement leads to resistance . to extinction , and 
(2) the ability to apjjly this^;pri,nciple to predict or explain particular 
^ situations. In teaching the concept, the instruction presents a verbal 
definition of 'the .cohcept, and sQUie examples of the concept These evehts. 
.are stored as episodes in episodic memory. A node is also initialized in 
semantic , memory , but all the node contains at this point are ^pointers to the. 
' ' episodes. The node can be accessed by the, name **priYiciple of intermittent 
reinforcement" and also from^the examples. Figure 6 illustrates this state 
of affairs. (This discussion assumes the learner is successful in acquir- 
ing all presented information, of course he'may fail.) The model holds that 
storage of epiosdes tied to a retrieval cue is the necessary precursor to 
the development of a node in semantic memory. However the episodes may be 
linguistic (presentation of a rule), descriptive (Verbal description of an 
example), or imaginal-sensory (presentation of actual experience)- For most 
real life cpncepts the initial episodes, are probably some combination of 
' " theseV' . ^ 

Now let us assume that the instruction asks the student acquiring the 
' principle of intermittent reinforcement a question about the definition of 

the principle. The student must. state the rulg. To accomplish this , the 
. student accesses the appropriate node through the name, he then attemjits to 
locate an episode that contains the verbalized rule, and then attemjits to 
repeat it. Note that the episode does Hot contain an exact coj)y of the rule, 
rather it c^ontains what the student has encoded when first given .the rule, * 
Assuming the student locates an appropriate episode, he operates upon the 
content of the episode to produce a verbalization. The operation upon 
< some contouts of episodic memory initializes a change in semantic memory : • 



a representation of the verbalization is entered fnto semantic memory/ If 
the 'subject were unable t^^ retrieve an episode in episodic memory that 
contained ^a ^verbalization of the concept, but was able to retrijeye 6ne or . 
• •more episodes containing presented examples of the concept, then the learner 
might apply reaso.ning to- the examples in order to construct a verbalization. 
In this case, a, representation of £he verbalization based on analysis of the ■ '. 
examples would be; stored :;^n semantic memory. In this latter case, however, 
the learner would have had to try to isolate some set of common features 
acrps$ the examples in order to develop a generalisation. This analysis 
might also lead to the storage in semantic^memory of a procedure for testing 
new instances to determine if they are examples or not. -This latter process 
is a form of what is called discovery lea^rning. Figure 7 illustrates the ^ • 
situation after the learner has produced a' verbalization. : 

The development of, a procedure for testing new examples can also be 
facilitated asking the learner' to recqgnize new examples, as in the Watts 
and Anderson (1971) study. Presented with an application question the learner 
must retrieve examples, isolate common features, develop a system for testing 
new examples, and apply it. to the presented new examples. This process also 
leads to the storage in semantic memory of a procedure for testing new exam- 
ples, and If successful would increase the learner's ability to recognize 
other new examples, l-igure 8 illustrates this state of affairs. It can be 
hoted that in this case the learner has acquired an ability to acquire the 
principle through expository learning. \ 

The procedure for testing instances that the subject develops will lie 
iLs simple or as complex as* necessary to integrate the presented examples and 
non-examples. If a simple rule will -relate the examples the leai:ner has 
stored in his episodic memory, then a simple rule will be stored in semantic 
memory. I. f the given examples are disparate^ and bear at host -what ^And(»rsoii 
.^'5, *LL tl^^'/^>) ^all n family relationship, t^en the catalogueil ( o;U i n^. 



' procedure will reflect this complexity. This liotion. that it is procedures 
for doing an act that are stored in semantic memory is-,iiQt new. Neisser (1967) 
has argued for such a position. Moreover the position has some important 
relationships Osgg^dyCigS^^^^^^^^o"^^^ tak^ n^^^ to be' 

the process that produce^ a response instead of a copy of a fesponse^i ; Finally 
of course the idea relate^ back to Bartlett> (1932) and James » (1892) ideas 
about memory. .. , 

If a learner is asked to engage in other types of tasks while learning 
a C(^ncept, ' principle, skill or other information the kinds of features laid\ 
down in semantic memory will reflect what the student has done with the * 
information. 'If the student is askedto relate the information to more 
inclusive, information (i.e., as ^ith advanced organizers) then subsumatofy 
connections will be formed. If the learner is required to evaluate certain 
kinds of information, then evaluative features (or schemata) will be t-ormed. 

The model that is being- proposed can now be summarized. (1) The learner 
is conceptualized as a complex information processing system. (2) The infor- 
mtition ]irocess lng system contains two types, of long term memory, episodic 
and semantic. Lipisodic memory contains r/jpresentat ions of the events encoun- 
tered by the system while semantic memory contains its generalized knowledge. 
There are intim^^e relations between semantic and episodic memory. (3) Most 
educational tasks require that changes he made in thp learner^s semantic' 
^memory. (4) Semantic memory is best represente^ by a network motlcl . The 
network contains nodes labeled directed connections between ^otles , ami 
rata lo'guctl procechires. (5) A new node, (and capability) is developed in 
semantic memory tlirough a two step process in which: a) rolovant opisoiics 
are laid down in episodic memory ami associateil with a notk\ label, and 
b) the learner attempts to perform some task involving the node and the 

flresults of the processing performetl in the task are connected to tlie notle 
In semantic moiiiory. (61 Because the results t)t operations i^erformed on a 



node are stored in semantic memory, semantic .memory is "in a continual state 

of evQlution as the learner us,esn6<le'^'^" in new ^tUcit^ions or contexts. 

l£ seems to 'me that a', model- along thelin.es suggested here serves to 

organize and lend insijght into a number of disparate areas within ^learning 

. " • ■ v.: •• ■ ' . • ■■ - • • ■ . 

and instructional ps)rci)Qlogy:^ First, the model does a^^ood job of handling ^ 

the effects of questions. ■ The model easily incorporates DAM by arguing that 

the ^,f/ects of broader questions during instruction lead to an enriched trace" 

in episodic memory. When the final retention task involves simple recall of- 

the^ presented information, this enriched trace provides for more recaXj. 

Such an enriched episodic trace wbuld", of ^course/ occur only* if the subject 

iimited processing td that material needed to handle the current task. 

The model can also handle the' effect s of higher level questions when 

higtier level outcome measures are employed: Such questions serve to develop 

the semantic memory representation al'ong certain linos.. Jn addition the 

■' . ■ , . . ^ , 

,\ . ' • ' ' ■■• ■ ' . . ■ 

Tuodei is not embarrassed by failures of higher level questions to. produce 
\ . ' ^ • . ^ 

higher level outcomes. The model argues for a large number of stages that 

must be completed if an adequate semantic memory representation of the 

concept is to be formed. A failure of processing can occur at any of these 

stages. Beyond simply ui lowing for failure, the model allows prediction of 

the .K Inds of failure that An occur. While these are numerous, they are 

also assQSsable providing a means for testing predictions from the model. 

(Since the nK^del argues that "the semantic memor^trace is dependent on 

t.lie nature of the instructional task, it can handle vffocts like those 

iliscusseil by Mayer (i^DVfV^ 11)76) in wh i ch 'd i f fcront instructional t reatiiKMit'^t> 

tha t- em[)ha:;.i .'.e different kiniU^ of tasks were able to lead to differential 

';u[)er ior i ty on near and far transfer. Tl\<^ model also hatidlos the kinds oi' 

meiiningful ness n^f feet s tlescribed by Rtiyewand his associated (Royer f, Clable, 



ef JsLe moilel refers to the learner*;* [)rii9^;- kBfcwleclge about the inlonuation 



Royer, IDT^"? Note !> ) . Mbanjng fulness 1 rom the viewpoint 
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being presented.- iVhat Rdyer's work has shown is when the learner has 

\* ' *■ ' 

little prior knowledge aftout a subject matter (Royer's complex abstract 

•' ' 

'passages) and the term's used in the passage access relatively empty nodes 
in. semantic memory, presenting prior information a'bjout these nodes enha>nces 
learning. According to the model, this should happen sincb^the encoding 
of the information presented in episodic memory will fi^e strongly influenced 
by how well the , subject can encode or interpret the prii^^sented language. 
The model would handle other kinds of meaningfulness effects (Johnson, 1975) 
similarly. , * , ;^ 

Another variable which should influence encoding of\episodes is imagery. 
Imagery has. been shown to enhance retention of presehted- itiformation (Paivio, 
1969, 1971, 1974; Anderson 5 Hidde, 1971). However the effects of imagery 
have typically been weaker when prose materials have been used. Moreover no 
study known to the author has even examined the effects of imagery when more 
than reproductive recall has been required. Under the model being outlined 
here, imagery would have its effect by influencing the ease with which epi- 
sodes can be remembered. Episodes which are more vivid are remembered better 
Thus, the model predicts that the effect of imagery on productive learning 
(learning that goes beyond the information given) and retention will be in- 
direct. Imagery effects episodic memory, but not semantic.. 

The model has some interesting things to say about discovery and exposi- 
tory teaching which were alluded to above. It says that by appropriately 
choo ng tasks both expository and discovery methods can lead to the same 
instructional outcomes. The methods differ in the timing of certain kinds 
of processing, but both methods can produce the'kinds of processing that 
lead to identical representations in semantic memory. More specifically, 
in order to learn a concept by discovery or by exposition, the learner must 
still process examples so as to construct a schema that allows him to test 
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new possible instances. Ccipparisions between specific discovery and 
expository procedure^ in terms of outcomes are not relevant to any general 
comparison of the methods, since an outcome difference merely means that 
the particula;]^ procedure used did not lead subjects to do the n*eccssary pro- 
have practica^ meaning however). A test of 



cessing (that finding would 



the methods, in general, would first have to show that the compared procedure 

produced identical cognitive outcomes, and then would compare the efficiency 

y ] ■ . 

of the methods . ' ' 

I think the model is quite congruent with laboratory research on 

! 

concept identification-construction. Current conceptions of that research 

. * 

emphasize the roles of hypothesis testing and changing during learning 
(Kintsch, 1971; Levine, 1966). In terms of the present model the hypothesis 
can be considered the output of the inferential-reasoning process applied 
to presented examples. This hypothesis is- modified as the system receives 
more information in a feedback episode, more examples, and has to make a 
response again. ' ■ 

The model 'also relates to the literature on feedback and an information- 
processing analysis of feedback effects in meaningful learning (Kulhavy, 
1976, Note 3 ; Phye, 1977, Note 6). The model suggests that the feedback 
episode will have an effect only- i^f the subject attempts to use it in sub- 
sequent responding to the problem. Very often this is true, certainly in 
most concept ^identification studies it is true. However in many' educational 
situations and in much of the DRE research the' learner is. 'certainly not 
instructionally led to re-respond after the feedback episode. The current 
model su^^ests that the effect of feedback will be greater if the learner 
were. 

The connections between episodic and semantic memories in the model 
give the system access to sensory features and make analysis on the basis 
of sensory features possible. Anderson (1975), Anderson and Ortony (1975) 
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and Walker (1975) have provided evidence of the necessity for such 
capabilities in the system. However the model makes some predictions that 
those authors did not consider*. Walker (1975) for example asked subjects 
to assess whether or not particular objects were in fact reasonable category 
members* Thirty pound turkeys were one of the examples used, which were on 
the borderline- of reasonableness. Subjects took ionger to'make judgments 
about 'items near the "borderlines than items distant. from them. Anderson 
(1975) has argued that this result suppojj^s an exemplar conception of judg- 
,ments. This is compatible with the present model. However, the present 
model goes on to predict that subjects might not use an exemplar, basis 
to perform the task a second time* The first task might have. effected - 
changes in semantic memory such that they encoded that 30 pound turkeys 
were unreasonable (or reasonable). 

The model that has been presented is certainly not a completed theory. 
Rather it is better conceptualized as a set of generalizations that need to 
bre formalized and made specifically operational. In addition the relationship 
between the current model and other current conceptions need to be explored 
more fuUX (e.g., Anderson, (^75; Pask, 1975; Scandura, 1977; Wittrock, 1974). 
The principle value of these assumptions is to serve as a set of guidelines 
for researctf on prose learning. The model leads to some general expectations 
about the^^kinds of effects that instructional manipulations can have. These 
expectation^*-can be formalized and tested in particular cases. In a sense, 
the ^mo^del I have presented is similar to a schema as discussed by Anderson, 
*et a]^ (1976), Rumelhart and Ortony (1976), and Schank (1975), It is really^ 
a kind of abstract plot which can be particularized in specific situations. 
As such it can, serve as a guid^ to future research, but that future research 



should* also serve to flesh out and make more specific the more vague areas 
of the model. , *^ 
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